2024-09-17T04:22:56.294 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-17T04:22:56.316 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T04:22:56.405 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908540 branch: wip-pdonnell-testing-20240916.200549-debug description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/alternate-pool} email: pdonnell@redhat.com first_in_suite: false job_id: '7908540' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240916.200549-debug ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: client: pid file: '' mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - bad backtrace - object missing on disk - error reading table object - error reading sessionmap - unmatched fragstat - unmatched rstat - was unreadable, recreating it now - Scrub error on inode - Metadata damage detected - MDS_FAILED - MDS_DAMAGE sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240916.200549-debug sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 476 sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf sleep_before_teardown: 0 subset: 320/512 suite: fs suite_branch: wip-pdonnell-testing-20240916.200549-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi037.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNy/At/D3by+/dEkkI/d45LiBQBqS6hLJSrxvosT7nQ34twYHz8GmAoOr+oKhAtE6gMJ0VL9j8Ozjqt5LlYMxgs= smithi092.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEc66UZbUU/s6SQrv/Lc7fz5B2DLhLKpQNexSMDp04LwzgcAKVrZ2W9OsDBPTemVtL2NWDEZvwiBwSYlgoV/69E= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_recovery_pool teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-17_02:09:51 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-17T04:22:56.405 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa; will attempt to use it 2024-09-17T04:22:56.406 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa/tasks 2024-09-17T04:22:56.406 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-17T04:22:56.408 INFO:teuthology.task.internal:Checking packages... 2024-09-17T04:22:56.429 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf' 2024-09-17T04:22:56.429 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-17T04:22:56.429 INFO:teuthology.packaging:ref: None 2024-09-17T04:22:56.429 INFO:teuthology.packaging:tag: None 2024-09-17T04:22:56.429 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240916.200549-debug 2024-09-17T04:22:56.429 INFO:teuthology.packaging:sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T04:22:56.429 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240916.200549-debug 2024-09-17T04:22:56.569 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4989.gce1b342b 2024-09-17T04:22:56.571 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-17T04:22:56.578 INFO:teuthology.task.internal:no buildpackages task found 2024-09-17T04:22:56.578 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-17T04:22:56.595 INFO:teuthology.task.internal:Saving configuration 2024-09-17T04:22:56.613 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-17T04:22:56.620 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-17T04:22:56.638 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908540', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 04:17:52.498917', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNy/At/D3by+/dEkkI/d45LiBQBqS6hLJSrxvosT7nQ34twYHz8GmAoOr+oKhAtE6gMJ0VL9j8Ozjqt5LlYMxgs='} 2024-09-17T04:22:56.658 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908540', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 04:17:52.500727', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEc66UZbUU/s6SQrv/Lc7fz5B2DLhLKpQNexSMDp04LwzgcAKVrZ2W9OsDBPTemVtL2NWDEZvwiBwSYlgoV/69E='} 2024-09-17T04:22:56.658 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-17T04:22:56.665 INFO:teuthology.task.internal:roles: ubuntu@smithi037.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-17T04:22:56.665 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-17T04:22:56.666 INFO:teuthology.run_tasks:Running task console_log... 2024-09-17T04:22:56.720 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f61ae933ac0>, signals=[15]) 2024-09-17T04:22:56.720 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-17T04:22:56.726 INFO:teuthology.task.internal:Opening connections... 2024-09-17T04:22:56.726 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi037.front.sepia.ceph.com 2024-09-17T04:22:56.727 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:22:56.805 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2024-09-17T04:22:56.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:22:56.881 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-17T04:22:56.891 DEBUG:teuthology.orchestra.run.smithi037:> uname -m 2024-09-17T04:22:56.909 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64 2024-09-17T04:22:56.909 DEBUG:teuthology.orchestra.run.smithi037:> cat /etc/os-release 2024-09-17T04:22:56.967 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="CentOS Stream" 2024-09-17T04:22:56.967 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="9" 2024-09-17T04:22:56.967 INFO:teuthology.orchestra.run.smithi037.stdout:ID="centos" 2024-09-17T04:22:56.967 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="rhel fedora" 2024-09-17T04:22:56.967 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="9" 2024-09-17T04:22:56.967 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el9" 2024-09-17T04:22:56.967 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T04:22:56.967 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31" 2024-09-17T04:22:56.968 INFO:teuthology.orchestra.run.smithi037.stdout:LOGO="fedora-logo-icon" 2024-09-17T04:22:56.968 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T04:22:56.968 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://centos.org/" 2024-09-17T04:22:56.968 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T04:22:56.968 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T04:22:56.968 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T04:22:56.969 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2024-09-17T04:22:57.001 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2024-09-17T04:22:57.019 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2024-09-17T04:22:57.020 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2024-09-17T04:22:57.079 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream" 2024-09-17T04:22:57.079 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="9" 2024-09-17T04:22:57.079 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2024-09-17T04:22:57.079 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2024-09-17T04:22:57.079 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="9" 2024-09-17T04:22:57.080 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el9" 2024-09-17T04:22:57.080 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T04:22:57.080 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2024-09-17T04:22:57.080 INFO:teuthology.orchestra.run.smithi092.stdout:LOGO="fedora-logo-icon" 2024-09-17T04:22:57.080 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T04:22:57.080 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/" 2024-09-17T04:22:57.080 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T04:22:57.081 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T04:22:57.081 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T04:22:57.081 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2024-09-17T04:22:57.110 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-17T04:22:57.122 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-17T04:22:57.128 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-17T04:22:57.129 DEBUG:teuthology.orchestra.run.smithi037:> test '!' -e /home/ubuntu/cephtest 2024-09-17T04:22:57.131 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2024-09-17T04:22:57.148 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-17T04:22:57.154 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-17T04:22:57.154 DEBUG:teuthology.orchestra.run.smithi037:> test -z $(ls -A /var/lib/ceph) 2024-09-17T04:22:57.189 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2024-09-17T04:22:57.271 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-17T04:22:57.324 INFO:teuthology.run_tasks:Running task kernel... 2024-09-17T04:22:57.338 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-17T04:22:57.338 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-17T04:22:57.339 DEBUG:teuthology.orchestra.run.smithi037:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T04:22:57.339 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T04:22:57.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T04:22:57.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T04:22:57.357 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2024-09-17T04:22:57.357 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-09-17T04:22:57.414 INFO:teuthology.orchestra.run.smithi092.stdout:5.14.0-503.el9.x86_64 2024-09-17T04:22:57.415 INFO:teuthology.task.kernel:Running kernel on smithi092: 5.14.0-503.el9.x86_64 2024-09-17T04:22:57.415 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum install -y kernel 2024-09-17T04:22:57.416 INFO:teuthology.orchestra.run.smithi037.stdout:5.14.0-503.el9.x86_64 2024-09-17T04:22:57.416 INFO:teuthology.task.kernel:Running kernel on smithi037: 5.14.0-503.el9.x86_64 2024-09-17T04:22:57.416 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum install -y kernel 2024-09-17T04:23:00.362 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 6.6 MB/s | 8.3 MB 00:01 2024-09-17T04:23:00.668 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 5.4 MB/s | 8.3 MB 00:01 2024-09-17T04:23:02.542 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-17T04:23:04.346 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 7.1 MB/s | 20 MB 00:02 2024-09-17T04:23:08.585 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.5 MB 00:02 2024-09-17T04:23:09.885 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.5 MB 00:01 2024-09-17T04:23:10.876 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 19 kB 00:01 2024-09-17T04:23:11.357 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 113 kB/s | 19 kB 00:00 2024-09-17T04:23:12.080 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-17T04:23:12.453 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-17T04:23:17.461 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-09-17T04:23:18.117 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-17T04:23:19.341 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T04:23:19.342 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T04:23:19.441 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-17T04:23:19.450 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-17T04:23:19.450 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2024-09-17T04:23:19.450 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-17T04:23:19.450 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-17T04:23:19.450 INFO:teuthology.orchestra.run.smithi092.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T04:23:19.451 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-17T04:23:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T04:23:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T04:23:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T04:23:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:23:19.451 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-17T04:23:19.451 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2024-09-17T04:23:19.451 INFO:teuthology.orchestra.run.smithi092.stdout:Install 4 Packages 2024-09-17T04:23:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:23:19.452 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 79 M 2024-09-17T04:23:19.452 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 126 M 2024-09-17T04:23:19.452 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-17T04:23:19.782 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 172.65.90.22) 2024-09-17T04:23:19.799 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 172.65.90.22) 2024-09-17T04:23:19.868 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 172.65.90.22) 2024-09-17T04:23:19.905 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 172.65.90.22) 2024-09-17T04:23:19.905 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 172.65.90.22) 2024-09-17T04:23:19.944 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 172.65.90.22) 2024-09-17T04:23:20.182 INFO:teuthology.orchestra.run.smithi037.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T04:23:20.182 INFO:teuthology.orchestra.run.smithi037.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T04:23:20.211 INFO:teuthology.orchestra.run.smithi092.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 71 kB/s | 36 kB 00:00 2024-09-17T04:23:20.259 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 172.65.90.22) 2024-09-17T04:23:20.277 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T04:23:20.288 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T04:23:20.289 INFO:teuthology.orchestra.run.smithi037.stdout: Package Architecture Version Repository Size 2024-09-17T04:23:20.289 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T04:23:20.289 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-09-17T04:23:20.289 INFO:teuthology.orchestra.run.smithi037.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T04:23:20.289 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-09-17T04:23:20.289 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T04:23:20.289 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T04:23:20.289 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T04:23:20.289 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:23:20.289 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-09-17T04:23:20.289 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T04:23:20.289 INFO:teuthology.orchestra.run.smithi037.stdout:Install 4 Packages 2024-09-17T04:23:20.289 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:23:20.290 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 79 M 2024-09-17T04:23:20.290 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 126 M 2024-09-17T04:23:20.290 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-09-17T04:23:20.318 INFO:teuthology.orchestra.run.smithi092.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 172.65.90.22) 2024-09-17T04:23:20.393 INFO:teuthology.orchestra.run.smithi037.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 437 kB/s | 36 kB 00:00 2024-09-17T04:23:20.768 INFO:teuthology.orchestra.run.smithi037.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-09-17T04:23:21.244 INFO:teuthology.orchestra.run.smithi037.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 37 MB/s | 35 MB 00:00 2024-09-17T04:23:21.586 INFO:teuthology.orchestra.run.smithi037.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 24 MB/s | 29 MB 00:01 2024-09-17T04:23:21.586 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-09-17T04:23:21.586 INFO:teuthology.orchestra.run.smithi037.stdout:Total 61 MB/s | 79 MB 00:01 2024-09-17T04:23:22.295 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-09-17T04:23:22.677 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-09-17T04:23:22.678 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-09-17T04:23:23.286 INFO:teuthology.orchestra.run.smithi092.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 4.4 MB/s | 16 MB 00:03 2024-09-17T04:23:23.628 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-09-17T04:23:23.628 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-09-17T04:23:26.180 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-09-17T04:23:26.881 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T04:23:26.970 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T04:23:29.703 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T04:23:29.840 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T04:23:32.526 INFO:teuthology.orchestra.run.smithi092.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 2.3 MB/s | 29 MB 00:12 2024-09-17T04:23:34.958 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T04:23:34.999 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T04:23:40.146 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T04:23:53.439 INFO:teuthology.orchestra.run.smithi092.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 1.0 MB/s | 35 MB 00:33 2024-09-17T04:23:53.441 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-17T04:23:53.441 INFO:teuthology.orchestra.run.smithi092.stdout:Total 2.3 MB/s | 79 MB 00:33 2024-09-17T04:23:54.127 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-17T04:23:54.489 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-17T04:23:54.490 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-17T04:23:55.455 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-17T04:23:55.456 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-17T04:23:57.879 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-17T04:23:58.617 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T04:23:58.697 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T04:24:01.436 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T04:24:01.568 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T04:24:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T04:24:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T04:24:11.246 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T04:24:18.948 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T04:24:18.990 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T04:24:19.369 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T04:24:19.369 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T04:24:19.370 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T04:24:19.370 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T04:24:20.229 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T04:24:20.229 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:24:20.229 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-09-17T04:24:20.229 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T04:24:20.229 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T04:24:20.230 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T04:24:20.230 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T04:24:20.230 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:24:20.230 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T04:24:20.587 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T04:24:20.682 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T04:24:20.682 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T04:24:20.682 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T04:24:20.683 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T04:24:20.683 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T04:24:20.683 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-17T04:24:20.683 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi037.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T04:24:20.684 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T04:24:20.742 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T04:24:20.744 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T04:24:20.744 DEBUG:teuthology.orchestra.run.smithi037:> sudo rpm -qi grub2-tools 2024-09-17T04:24:20.775 INFO:teuthology.orchestra.run.smithi037.stdout:Name : grub2-tools 2024-09-17T04:24:20.776 INFO:teuthology.orchestra.run.smithi037.stdout:Epoch : 1 2024-09-17T04:24:20.776 INFO:teuthology.orchestra.run.smithi037.stdout:Version : 2.06 2024-09-17T04:24:20.776 INFO:teuthology.orchestra.run.smithi037.stdout:Release : 82.el9 2024-09-17T04:24:20.776 INFO:teuthology.orchestra.run.smithi037.stdout:Architecture: x86_64 2024-09-17T04:24:20.776 INFO:teuthology.orchestra.run.smithi037.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T04:24:20.776 INFO:teuthology.orchestra.run.smithi037.stdout:Group : Unspecified 2024-09-17T04:24:20.776 INFO:teuthology.orchestra.run.smithi037.stdout:Size : 8274670 2024-09-17T04:24:20.776 INFO:teuthology.orchestra.run.smithi037.stdout:License : GPLv3+ 2024-09-17T04:24:20.776 INFO:teuthology.orchestra.run.smithi037.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T04:24:20.776 INFO:teuthology.orchestra.run.smithi037.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T04:24:20.776 INFO:teuthology.orchestra.run.smithi037.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T04:24:20.776 INFO:teuthology.orchestra.run.smithi037.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T04:24:20.776 INFO:teuthology.orchestra.run.smithi037.stdout:Packager : builder@centos.org 2024-09-17T04:24:20.777 INFO:teuthology.orchestra.run.smithi037.stdout:Vendor : CentOS 2024-09-17T04:24:20.777 INFO:teuthology.orchestra.run.smithi037.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T04:24:20.777 INFO:teuthology.orchestra.run.smithi037.stdout:Summary : Support tools for GRUB. 2024-09-17T04:24:20.777 INFO:teuthology.orchestra.run.smithi037.stdout:Description : 2024-09-17T04:24:20.777 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:24:20.777 INFO:teuthology.orchestra.run.smithi037.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T04:24:20.777 INFO:teuthology.orchestra.run.smithi037.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T04:24:20.777 INFO:teuthology.orchestra.run.smithi037.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T04:24:20.777 INFO:teuthology.orchestra.run.smithi037.stdout:hardware devices. 2024-09-17T04:24:20.777 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:24:20.777 INFO:teuthology.orchestra.run.smithi037.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T04:24:20.778 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T04:24:20.778 INFO:teuthology.task.kernel:Updating grub on smithi037 to boot 5.14.0-508.el9.x86_64 2024-09-17T04:24:20.778 DEBUG:teuthology.orchestra.run.smithi037:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T04:24:21.449 INFO:teuthology.orchestra.run.smithi037.stderr:Generating grub configuration file ... 2024-09-17T04:24:22.751 INFO:teuthology.orchestra.run.smithi037.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T04:24:22.780 INFO:teuthology.orchestra.run.smithi037.stderr:done 2024-09-17T04:24:22.782 DEBUG:teuthology.orchestra.run.smithi037:> mktemp 2024-09-17T04:24:22.797 INFO:teuthology.orchestra.run.smithi037.stdout:/tmp/tmp.mZ9Ewwa8ah 2024-09-17T04:24:22.797 DEBUG:teuthology.orchestra.run.smithi037:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.mZ9Ewwa8ah 2024-09-17T04:24:22.862 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0666 /tmp/tmp.mZ9Ewwa8ah 2024-09-17T04:24:22.998 DEBUG:teuthology.orchestra.remote:smithi037:/tmp/tmp.mZ9Ewwa8ah is 6KB 2024-09-17T04:24:23.046 DEBUG:teuthology.orchestra.run.smithi037:> rm -fr /tmp/tmp.mZ9Ewwa8ah 2024-09-17T04:24:23.061 DEBUG:teuthology.orchestra.run.smithi037:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T04:24:23.123 INFO:teuthology.orchestra.run.smithi037.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T04:24:23.124 INFO:teuthology.orchestra.run.smithi037.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T04:24:23.124 INFO:teuthology.orchestra.run.smithi037.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T04:24:23.124 INFO:teuthology.orchestra.run.smithi037.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T04:24:23.125 DEBUG:teuthology.orchestra.run.smithi037:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T04:24:23.275 DEBUG:teuthology.orchestra.run.smithi037:> sudo shutdown -r now 2024-09-17T04:24:49.580 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T04:24:49.617 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T04:24:50.001 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T04:24:50.001 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T04:24:50.001 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T04:24:50.002 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T04:24:50.806 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T04:24:50.806 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:24:50.806 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-17T04:24:50.806 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T04:24:50.806 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T04:24:50.806 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T04:24:50.806 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T04:24:50.807 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:24:50.807 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-17T04:24:51.155 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T04:24:51.235 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T04:24:51.235 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T04:24:51.236 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T04:24:51.236 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T04:24:51.236 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T04:24:51.236 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-17T04:24:51.236 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi092.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T04:24:51.237 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T04:24:51.295 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T04:24:51.295 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T04:24:51.296 DEBUG:teuthology.orchestra.run.smithi092:> sudo rpm -qi grub2-tools 2024-09-17T04:24:51.327 INFO:teuthology.orchestra.run.smithi092.stdout:Name : grub2-tools 2024-09-17T04:24:51.327 INFO:teuthology.orchestra.run.smithi092.stdout:Epoch : 1 2024-09-17T04:24:51.327 INFO:teuthology.orchestra.run.smithi092.stdout:Version : 2.06 2024-09-17T04:24:51.327 INFO:teuthology.orchestra.run.smithi092.stdout:Release : 82.el9 2024-09-17T04:24:51.327 INFO:teuthology.orchestra.run.smithi092.stdout:Architecture: x86_64 2024-09-17T04:24:51.328 INFO:teuthology.orchestra.run.smithi092.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T04:24:51.328 INFO:teuthology.orchestra.run.smithi092.stdout:Group : Unspecified 2024-09-17T04:24:51.328 INFO:teuthology.orchestra.run.smithi092.stdout:Size : 8274670 2024-09-17T04:24:51.328 INFO:teuthology.orchestra.run.smithi092.stdout:License : GPLv3+ 2024-09-17T04:24:51.328 INFO:teuthology.orchestra.run.smithi092.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T04:24:51.328 INFO:teuthology.orchestra.run.smithi092.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T04:24:51.328 INFO:teuthology.orchestra.run.smithi092.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T04:24:51.328 INFO:teuthology.orchestra.run.smithi092.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T04:24:51.328 INFO:teuthology.orchestra.run.smithi092.stdout:Packager : builder@centos.org 2024-09-17T04:24:51.329 INFO:teuthology.orchestra.run.smithi092.stdout:Vendor : CentOS 2024-09-17T04:24:51.329 INFO:teuthology.orchestra.run.smithi092.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T04:24:51.329 INFO:teuthology.orchestra.run.smithi092.stdout:Summary : Support tools for GRUB. 2024-09-17T04:24:51.329 INFO:teuthology.orchestra.run.smithi092.stdout:Description : 2024-09-17T04:24:51.329 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:24:51.329 INFO:teuthology.orchestra.run.smithi092.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T04:24:51.329 INFO:teuthology.orchestra.run.smithi092.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T04:24:51.329 INFO:teuthology.orchestra.run.smithi092.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T04:24:51.329 INFO:teuthology.orchestra.run.smithi092.stdout:hardware devices. 2024-09-17T04:24:51.330 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:24:51.330 INFO:teuthology.orchestra.run.smithi092.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T04:24:51.331 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T04:24:51.331 INFO:teuthology.task.kernel:Updating grub on smithi092 to boot 5.14.0-508.el9.x86_64 2024-09-17T04:24:51.331 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T04:24:51.940 INFO:teuthology.orchestra.run.smithi092.stderr:Generating grub configuration file ... 2024-09-17T04:24:53.277 INFO:teuthology.orchestra.run.smithi092.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T04:24:53.278 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-17T04:24:53.279 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-09-17T04:24:53.280 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:24:53.316 INFO:teuthology.orchestra.run.smithi092.stderr:done 2024-09-17T04:24:53.318 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2024-09-17T04:24:53.334 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.VE8AMcRmaX 2024-09-17T04:24:53.334 DEBUG:teuthology.orchestra.run.smithi092:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.VE8AMcRmaX 2024-09-17T04:24:53.411 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.VE8AMcRmaX 2024-09-17T04:24:53.556 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.VE8AMcRmaX is 6KB 2024-09-17T04:24:53.576 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.VE8AMcRmaX 2024-09-17T04:24:53.590 DEBUG:teuthology.orchestra.run.smithi092:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T04:24:53.653 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T04:24:53.653 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T04:24:53.653 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T04:24:53.653 INFO:teuthology.orchestra.run.smithi092.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T04:24:53.654 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T04:24:53.809 DEBUG:teuthology.orchestra.run.smithi092:> sudo shutdown -r now 2024-09-17T04:25:11.741 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.37 2024-09-17T04:25:20.749 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-09-17T04:25:20.750 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:25:23.814 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-17T04:25:23.814 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-17T04:25:23.815 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:25:39.137 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.37 2024-09-17T04:25:42.205 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-09-17T04:25:51.145 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-09-17T04:25:51.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:25:51.205 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-17T04:25:51.206 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:25:52.237 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-09-17T04:25:52.712 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-09-17T04:25:52.712 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-17T04:25:52.712 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2024-09-17T04:25:52.727 INFO:teuthology.orchestra.run.smithi037.stdout:5.14.0-508.el9.x86_64 2024-09-17T04:25:52.727 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T04:25:52.727 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T04:25:52.727 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T04:25:53.728 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-17T04:25:53.728 DEBUG:teuthology.orchestra.run.smithi037:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T04:25:53.811 INFO:teuthology.orchestra.run.smithi037.stdout:ttyS1 2024-09-17T04:25:53.828 DEBUG:teuthology.parallel:result is None 2024-09-17T04:25:54.269 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-09-17T04:26:06.270 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-17T04:26:06.271 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:26:24.701 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-09-17T04:26:39.702 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-17T04:26:39.703 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:26:40.073 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-09-17T04:26:40.474 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-17T04:26:40.474 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-17T04:26:40.474 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-09-17T04:26:40.489 INFO:teuthology.orchestra.run.smithi092.stdout:5.14.0-508.el9.x86_64 2024-09-17T04:26:40.489 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T04:26:40.489 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T04:26:40.489 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T04:26:41.490 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-17T04:26:41.490 DEBUG:teuthology.orchestra.run.smithi092:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T04:26:41.583 INFO:teuthology.orchestra.run.smithi092.stdout:ttyS1 2024-09-17T04:26:41.604 DEBUG:teuthology.parallel:result is None 2024-09-17T04:26:41.605 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-17T04:26:41.613 INFO:teuthology.task.internal:Creating test directory... 2024-09-17T04:26:41.616 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T04:26:41.620 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T04:26:41.639 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-17T04:26:41.646 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-17T04:26:41.654 INFO:teuthology.task.internal:Creating archive directory... 2024-09-17T04:26:41.654 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T04:26:41.682 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T04:26:41.725 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-17T04:26:41.732 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-17T04:26:41.732 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T04:26:41.750 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T04:26:41.784 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T04:26:41.796 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T04:26:41.800 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T04:26:41.819 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T04:26:41.821 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-17T04:26:41.828 INFO:teuthology.task.internal:Configuring sudo... 2024-09-17T04:26:41.828 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T04:26:41.841 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T04:26:41.886 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-17T04:26:41.895 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-17T04:26:41.895 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T04:26:41.910 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T04:26:41.942 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T04:26:42.011 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T04:26:42.065 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T04:26:42.149 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T04:26:42.191 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:26:42.191 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T04:26:42.258 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T04:26:42.301 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T04:26:42.360 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T04:26:42.445 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T04:26:42.485 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:26:42.485 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T04:26:42.549 DEBUG:teuthology.orchestra.run.smithi037:> sudo service rsyslog restart 2024-09-17T04:26:42.551 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2024-09-17T04:26:42.598 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T04:26:42.754 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T04:26:43.105 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-17T04:26:43.113 INFO:teuthology.task.internal:Starting timer... 2024-09-17T04:26:43.113 INFO:teuthology.run_tasks:Running task pcp... 2024-09-17T04:26:43.124 INFO:teuthology.run_tasks:Running task selinux... 2024-09-17T04:26:43.133 DEBUG:teuthology.orchestra.run.smithi037:> sudo service auditd rotate 2024-09-17T04:26:43.197 INFO:teuthology.orchestra.run.smithi037.stdout:Rotating logs: 2024-09-17T04:26:43.199 DEBUG:teuthology.orchestra.run.smithi092:> sudo service auditd rotate 2024-09-17T04:26:43.306 INFO:teuthology.orchestra.run.smithi092.stdout:Rotating logs: 2024-09-17T04:26:43.312 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-17T04:26:43.312 DEBUG:teuthology.orchestra.run.smithi037:> /usr/sbin/getenforce 2024-09-17T04:26:43.336 INFO:teuthology.orchestra.run.smithi037.stdout:Permissive 2024-09-17T04:26:43.336 DEBUG:teuthology.orchestra.run.smithi092:> /usr/sbin/getenforce 2024-09-17T04:26:43.395 INFO:teuthology.orchestra.run.smithi092.stdout:Permissive 2024-09-17T04:26:43.395 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi037.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi092.front.sepia.ceph.com': 'permissive'} 2024-09-17T04:26:43.396 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T04:26:43.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T04:26:43.425 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T04:26:43.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T04:26:43.484 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-17T04:26:43.485 DEBUG:teuthology.orchestra.run.smithi037:> sudo /usr/sbin/setenforce permissive 2024-09-17T04:26:43.522 DEBUG:teuthology.orchestra.run.smithi092:> sudo /usr/sbin/setenforce permissive 2024-09-17T04:26:43.617 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-17T04:26:43.627 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-17T04:26:43.638 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-17T04:26:43.639 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi037.front.sepia.ceph.com,smithi092.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-17T04:30:31.906 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi037.front.sepia.ceph.com'), Remote(name='ubuntu@smithi092.front.sepia.ceph.com')] 2024-09-17T04:30:31.908 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-09-17T04:30:31.909 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:30:31.986 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-09-17T04:30:32.065 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-09-17T04:30:32.065 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-17T04:30:32.066 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:30:32.141 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-09-17T04:30:32.223 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-17T04:30:32.224 INFO:teuthology.run_tasks:Running task clock... 2024-09-17T04:30:32.233 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-17T04:30:32.233 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T04:30:32.234 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T04:30:32.236 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T04:30:32.236 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T04:30:32.265 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T04:30:32.280 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T04:30:32.293 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T04:30:32.305 INFO:teuthology.orchestra.run.smithi037.stderr:sudo: ntpd: command not found 2024-09-17T04:30:32.306 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T04:30:32.317 INFO:teuthology.orchestra.run.smithi037.stdout:506 Cannot talk to daemon 2024-09-17T04:30:32.330 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: ntpd: command not found 2024-09-17T04:30:32.332 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T04:30:32.341 INFO:teuthology.orchestra.run.smithi092.stdout:506 Cannot talk to daemon 2024-09-17T04:30:32.347 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T04:30:32.355 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T04:30:32.369 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T04:30:32.406 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-09-17T04:30:32.410 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T04:30:32.410 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-09-17T04:30:32.410 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T04:30:32.410 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T04:30:32.410 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T04:30:32.410 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T04:30:32.427 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-09-17T04:30:32.430 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T04:30:32.430 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-09-17T04:30:32.430 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T04:30:32.430 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T04:30:32.430 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T04:30:32.430 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T04:30:32.431 INFO:teuthology.run_tasks:Running task install... 2024-09-17T04:30:32.438 DEBUG:teuthology.task.install:project ceph 2024-09-17T04:30:32.439 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}} 2024-09-17T04:30:32.439 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'} 2024-09-17T04:30:32.439 INFO:teuthology.task.install:Using flavor: default 2024-09-17T04:30:32.444 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-17T04:30:32.444 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-17T04:30:32.445 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf', 'tag': None, 'wait_for_package': False} 2024-09-17T04:30:32.445 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T04:30:32.447 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf', 'tag': None, 'wait_for_package': False} 2024-09-17T04:30:32.447 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T04:30:32.593 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/centos/9/flavors/default/ 2024-09-17T04:30:32.593 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4989.gce1b342b 2024-09-17T04:30:32.597 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/centos/9/flavors/default/ 2024-09-17T04:30:32.597 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4989.gce1b342b 2024-09-17T04:30:32.704 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T04:30:32.705 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:30:32.705 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T04:30:32.710 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T04:30:32.710 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:30:32.711 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T04:30:32.733 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-17T04:30:32.733 DEBUG:teuthology.orchestra.run.smithi037:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T04:30:32.736 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-17T04:30:32.736 DEBUG:teuthology.orchestra.run.smithi092:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T04:30:32.800 DEBUG:teuthology.orchestra.run.smithi037:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T04:30:32.802 DEBUG:teuthology.orchestra.run.smithi092:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T04:30:32.877 DEBUG:teuthology.orchestra.run.smithi037:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T04:30:32.879 DEBUG:teuthology.orchestra.run.smithi092:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T04:30:32.942 INFO:teuthology.orchestra.run.smithi037.stdout:check_obsoletes = 1 2024-09-17T04:30:32.943 INFO:teuthology.orchestra.run.smithi092.stdout:check_obsoletes = 1 2024-09-17T04:30:32.945 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum clean all 2024-09-17T04:30:32.946 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum clean all 2024-09-17T04:30:33.181 INFO:teuthology.orchestra.run.smithi037.stdout:45 files removed 2024-09-17T04:30:33.186 INFO:teuthology.orchestra.run.smithi092.stdout:45 files removed 2024-09-17T04:30:33.208 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-17T04:30:33.213 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-17T04:30:33.769 INFO:teuthology.orchestra.run.smithi092.stdout:ceph packages for x86_64 313 kB/s | 84 kB 00:00 2024-09-17T04:30:33.782 INFO:teuthology.orchestra.run.smithi037.stdout:ceph packages for x86_64 310 kB/s | 84 kB 00:00 2024-09-17T04:30:34.061 INFO:teuthology.orchestra.run.smithi092.stdout:ceph noarch packages 98 kB/s | 24 kB 00:00 2024-09-17T04:30:34.074 INFO:teuthology.orchestra.run.smithi037.stdout:ceph noarch packages 98 kB/s | 24 kB 00:00 2024-09-17T04:30:34.337 INFO:teuthology.orchestra.run.smithi092.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-17T04:30:34.358 INFO:teuthology.orchestra.run.smithi037.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-17T04:30:34.978 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.3 MB 00:00 2024-09-17T04:30:35.108 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.3 MB 00:00 2024-09-17T04:30:36.851 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 32 MB/s | 20 MB 00:00 2024-09-17T04:30:38.115 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 9.8 MB/s | 20 MB 00:02 2024-09-17T04:30:42.274 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 17 MB/s | 6.5 MB 00:00 2024-09-17T04:30:43.704 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 2.2 MB/s | 6.5 MB 00:02 2024-09-17T04:30:43.791 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 68 kB/s | 19 kB 00:00 2024-09-17T04:30:45.228 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 17 MB/s | 23 MB 00:01 2024-09-17T04:30:45.253 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 83 kB/s | 19 kB 00:00 2024-09-17T04:30:46.816 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 16 MB/s | 23 MB 00:01 2024-09-17T04:30:50.625 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-17T04:30:52.480 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-17T04:30:52.835 INFO:teuthology.orchestra.run.smithi092.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-17T04:30:52.836 INFO:teuthology.orchestra.run.smithi092.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-17T04:30:52.848 INFO:teuthology.orchestra.run.smithi092.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-17T04:30:52.849 INFO:teuthology.orchestra.run.smithi092.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-17T04:30:52.849 INFO:teuthology.orchestra.run.smithi092.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-17T04:30:52.850 INFO:teuthology.orchestra.run.smithi092.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-17T04:30:52.852 INFO:teuthology.orchestra.run.smithi092.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-17T04:30:52.854 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-17T04:30:52.855 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-17T04:30:52.918 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-17T04:30:52.944 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-09-17T04:30:52.944 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-17T04:30:52.944 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-09-17T04:30:52.944 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-17T04:30:52.944 INFO:teuthology.orchestra.run.smithi092.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-17T04:30:52.944 INFO:teuthology.orchestra.run.smithi092.stdout: ceph x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 6.6 k 2024-09-17T04:30:52.944 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 5.5 M 2024-09-17T04:30:52.944 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 873 k 2024-09-17T04:30:52.944 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 148 k 2024-09-17T04:30:52.945 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 1.5 M 2024-09-17T04:30:52.945 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 154 k 2024-09-17T04:30:52.945 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 14 M 2024-09-17T04:30:52.945 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 7.4 M 2024-09-17T04:30:52.945 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 49 k 2024-09-17T04:30:52.945 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 10 M 2024-09-17T04:30:52.945 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 40 M 2024-09-17T04:30:52.945 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 285 k 2024-09-17T04:30:52.945 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 784 k 2024-09-17T04:30:52.945 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 225 k 2024-09-17T04:30:52.945 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-shell x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 27 k 2024-09-17T04:30:52.945 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 18 k 2024-09-17T04:30:52.946 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-17T04:30:52.946 INFO:teuthology.orchestra.run.smithi092.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-17T04:30:52.946 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-17T04:30:52.946 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 32 k 2024-09-17T04:30:52.946 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 766 k 2024-09-17T04:30:52.946 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 127 k 2024-09-17T04:30:52.946 INFO:teuthology.orchestra.run.smithi092.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-17T04:30:52.946 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-17T04:30:52.947 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-17T04:30:52.947 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-17T04:30:52.947 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-17T04:30:52.947 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-17T04:30:52.947 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 162 k 2024-09-17T04:30:52.947 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 321 k 2024-09-17T04:30:52.947 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 305 k 2024-09-17T04:30:52.947 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 99 k 2024-09-17T04:30:52.947 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 86 k 2024-09-17T04:30:52.948 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 3.1 M 2024-09-17T04:30:52.948 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 170 k 2024-09-17T04:30:52.948 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-17T04:30:52.948 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2024-09-17T04:30:52.948 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-17T04:30:52.948 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-17T04:30:52.948 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-17T04:30:52.948 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-17T04:30:52.948 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 3.4 M 2024-09-17T04:30:52.948 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 3.2 M 2024-09-17T04:30:52.948 INFO:teuthology.orchestra.run.smithi092.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-17T04:30:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-17T04:30:52.949 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-17T04:30:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-17T04:30:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-17T04:30:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T04:30:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 20 M 2024-09-17T04:30:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 34 k 2024-09-17T04:30:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 2.3 M 2024-09-17T04:30:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 291 k 2024-09-17T04:30:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 5.0 M 2024-09-17T04:30:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 18 M 2024-09-17T04:30:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 16 k 2024-09-17T04:30:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 25 k 2024-09-17T04:30:52.949 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T04:30:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T04:30:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T04:30:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T04:30:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T04:30:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-17T04:30:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T04:30:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T04:30:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 159 k 2024-09-17T04:30:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T04:30:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-17T04:30:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T04:30:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T04:30:52.950 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T04:30:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T04:30:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 507 k 2024-09-17T04:30:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T04:30:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 5.3 M 2024-09-17T04:30:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T04:30:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-17T04:30:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T04:30:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-17T04:30:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-17T04:30:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-17T04:30:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T04:30:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T04:30:52.951 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T04:30:52.952 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-17T04:30:52.952 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-17T04:30:52.952 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-17T04:30:52.952 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T04:30:52.952 INFO:teuthology.orchestra.run.smithi092.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-17T04:30:52.952 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T04:30:52.952 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T04:30:52.952 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T04:30:52.952 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 45 k 2024-09-17T04:30:52.952 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 143 k 2024-09-17T04:30:52.952 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T04:30:52.952 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T04:30:52.952 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T04:30:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T04:30:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-17T04:30:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-17T04:30:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T04:30:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T04:30:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T04:30:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-17T04:30:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-17T04:30:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T04:30:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T04:30:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-17T04:30:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T04:30:52.953 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T04:30:52.954 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T04:30:52.954 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T04:30:52.954 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T04:30:52.954 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T04:30:52.954 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T04:30:52.954 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T04:30:52.954 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T04:30:52.954 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T04:30:52.954 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T04:30:52.954 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T04:30:52.954 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T04:30:52.954 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T04:30:52.954 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T04:30:52.954 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T04:30:52.955 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T04:30:52.955 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T04:30:52.955 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T04:30:52.955 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-17T04:30:52.955 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T04:30:52.955 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T04:30:52.955 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T04:30:52.955 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T04:30:52.955 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-17T04:30:52.955 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T04:30:52.955 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T04:30:52.955 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T04:30:52.956 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T04:30:52.956 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T04:30:52.956 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T04:30:52.956 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T04:30:52.956 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T04:30:52.956 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T04:30:52.956 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T04:30:52.956 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T04:30:52.956 INFO:teuthology.orchestra.run.smithi092.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-17T04:30:52.956 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T04:30:52.956 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T04:30:52.956 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T04:30:52.956 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T04:30:52.957 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-17T04:30:52.957 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-17T04:30:52.957 INFO:teuthology.orchestra.run.smithi092.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T04:30:52.957 INFO:teuthology.orchestra.run.smithi092.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T04:30:52.957 INFO:teuthology.orchestra.run.smithi092.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T04:30:52.957 INFO:teuthology.orchestra.run.smithi092.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-17T04:30:52.957 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T04:30:52.957 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-17T04:30:52.957 INFO:teuthology.orchestra.run.smithi092.stdout:Installing weak dependencies: 2024-09-17T04:30:52.957 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-17T04:30:52.957 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:30:52.957 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-17T04:30:52.958 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-09-17T04:30:52.958 INFO:teuthology.orchestra.run.smithi092.stdout:Install 149 Packages 2024-09-17T04:30:52.958 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 8 Packages 2024-09-17T04:30:52.958 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:30:52.958 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 228 M 2024-09-17T04:30:52.958 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-17T04:30:53.885 INFO:teuthology.orchestra.run.smithi092.stdout:(1/157): ceph-19.3.0-4989.gce1b342b.el9.x86_64. 49 kB/s | 6.6 kB 00:00 2024-09-17T04:30:54.093 INFO:teuthology.orchestra.run.smithi092.stdout:(2/157): ceph-fuse-19.3.0-4989.gce1b342b.el9.x8 4.1 MB/s | 873 kB 00:00 2024-09-17T04:30:54.168 INFO:teuthology.orchestra.run.smithi092.stdout:(3/157): ceph-immutable-object-cache-19.3.0-498 1.9 MB/s | 148 kB 00:00 2024-09-17T04:30:54.235 INFO:teuthology.orchestra.run.smithi092.stdout:(4/157): ceph-base-19.3.0-4989.gce1b342b.el9.x8 11 MB/s | 5.5 MB 00:00 2024-09-17T04:30:54.361 INFO:teuthology.orchestra.run.smithi092.stdout:(5/157): ceph-mgr-19.3.0-4989.gce1b342b.el9.x86 12 MB/s | 1.5 MB 00:00 2024-09-17T04:30:54.503 INFO:teuthology.orchestra.run.smithi092.stdout:(6/157): ceph-mds-19.3.0-4989.gce1b342b.el9.x86 6.8 MB/s | 2.3 MB 00:00 2024-09-17T04:30:54.645 INFO:teuthology.orchestra.run.smithi092.stdout:(7/157): ceph-mon-19.3.0-4989.gce1b342b.el9.x86 17 MB/s | 5.0 MB 00:00 2024-09-17T04:30:54.757 INFO:teuthology.orchestra.run.smithi037.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-17T04:30:54.758 INFO:teuthology.orchestra.run.smithi037.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-17T04:30:54.769 INFO:teuthology.orchestra.run.smithi037.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-17T04:30:54.770 INFO:teuthology.orchestra.run.smithi037.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-17T04:30:54.771 INFO:teuthology.orchestra.run.smithi037.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-17T04:30:54.772 INFO:teuthology.orchestra.run.smithi037.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-17T04:30:54.774 INFO:teuthology.orchestra.run.smithi037.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-17T04:30:54.776 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-17T04:30:54.777 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-17T04:30:54.843 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T04:30:54.845 INFO:teuthology.orchestra.run.smithi092.stdout:(8/157): ceph-common-19.3.0-4989.gce1b342b.el9. 18 MB/s | 20 MB 00:01 2024-09-17T04:30:54.868 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-09-17T04:30:54.868 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-09-17T04:30:54.868 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-09-17T04:30:54.868 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-09-17T04:30:54.868 INFO:teuthology.orchestra.run.smithi037.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-17T04:30:54.868 INFO:teuthology.orchestra.run.smithi037.stdout: ceph x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 6.6 k 2024-09-17T04:30:54.868 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 5.5 M 2024-09-17T04:30:54.868 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 873 k 2024-09-17T04:30:54.868 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 148 k 2024-09-17T04:30:54.869 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 1.5 M 2024-09-17T04:30:54.869 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 154 k 2024-09-17T04:30:54.869 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 14 M 2024-09-17T04:30:54.869 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 7.4 M 2024-09-17T04:30:54.869 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 49 k 2024-09-17T04:30:54.869 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 10 M 2024-09-17T04:30:54.869 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 40 M 2024-09-17T04:30:54.869 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 285 k 2024-09-17T04:30:54.869 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 784 k 2024-09-17T04:30:54.869 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 225 k 2024-09-17T04:30:54.869 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 27 k 2024-09-17T04:30:54.869 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 18 k 2024-09-17T04:30:54.870 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-17T04:30:54.870 INFO:teuthology.orchestra.run.smithi037.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-17T04:30:54.870 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-17T04:30:54.870 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 32 k 2024-09-17T04:30:54.870 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 766 k 2024-09-17T04:30:54.870 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 127 k 2024-09-17T04:30:54.870 INFO:teuthology.orchestra.run.smithi037.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-17T04:30:54.870 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-17T04:30:54.870 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-17T04:30:54.870 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-17T04:30:54.870 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-17T04:30:54.870 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-17T04:30:54.871 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 162 k 2024-09-17T04:30:54.871 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 321 k 2024-09-17T04:30:54.871 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 305 k 2024-09-17T04:30:54.871 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 99 k 2024-09-17T04:30:54.871 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 86 k 2024-09-17T04:30:54.871 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 3.1 M 2024-09-17T04:30:54.871 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 170 k 2024-09-17T04:30:54.871 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-17T04:30:54.871 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrading: 2024-09-17T04:30:54.871 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-17T04:30:54.871 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-17T04:30:54.871 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-17T04:30:54.871 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-17T04:30:54.871 INFO:teuthology.orchestra.run.smithi037.stdout: librados2 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 3.4 M 2024-09-17T04:30:54.872 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 3.2 M 2024-09-17T04:30:54.872 INFO:teuthology.orchestra.run.smithi037.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-17T04:30:54.872 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-17T04:30:54.872 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-09-17T04:30:54.872 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-17T04:30:54.872 INFO:teuthology.orchestra.run.smithi037.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-17T04:30:54.872 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T04:30:54.872 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 20 M 2024-09-17T04:30:54.872 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 34 k 2024-09-17T04:30:54.872 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 2.3 M 2024-09-17T04:30:54.872 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 291 k 2024-09-17T04:30:54.872 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 5.0 M 2024-09-17T04:30:54.873 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 18 M 2024-09-17T04:30:54.873 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 16 k 2024-09-17T04:30:54.873 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 25 k 2024-09-17T04:30:54.873 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T04:30:54.873 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T04:30:54.873 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T04:30:54.873 INFO:teuthology.orchestra.run.smithi037.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T04:30:54.873 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T04:30:54.873 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-17T04:30:54.874 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T04:30:54.874 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T04:30:54.874 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 159 k 2024-09-17T04:30:54.874 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T04:30:54.874 INFO:teuthology.orchestra.run.smithi037.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-17T04:30:54.874 INFO:teuthology.orchestra.run.smithi037.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T04:30:54.874 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T04:30:54.874 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T04:30:54.874 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T04:30:54.874 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 507 k 2024-09-17T04:30:54.874 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T04:30:54.874 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 5.3 M 2024-09-17T04:30:54.874 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T04:30:54.875 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-17T04:30:54.875 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T04:30:54.875 INFO:teuthology.orchestra.run.smithi037.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-17T04:30:54.875 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-17T04:30:54.875 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-17T04:30:54.875 INFO:teuthology.orchestra.run.smithi037.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T04:30:54.875 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T04:30:54.875 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T04:30:54.875 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-17T04:30:54.875 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-17T04:30:54.875 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-17T04:30:54.875 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T04:30:54.875 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-17T04:30:54.875 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T04:30:54.876 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T04:30:54.876 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T04:30:54.876 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 45 k 2024-09-17T04:30:54.876 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 143 k 2024-09-17T04:30:54.876 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T04:30:54.876 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T04:30:54.876 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T04:30:54.876 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T04:30:54.876 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-17T04:30:54.876 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-17T04:30:54.876 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T04:30:54.876 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T04:30:54.877 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T04:30:54.877 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-17T04:30:54.877 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-17T04:30:54.877 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T04:30:54.877 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T04:30:54.877 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-17T04:30:54.877 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T04:30:54.877 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T04:30:54.877 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T04:30:54.877 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T04:30:54.877 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T04:30:54.877 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T04:30:54.877 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T04:30:54.878 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T04:30:54.878 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T04:30:54.878 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T04:30:54.878 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T04:30:54.878 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T04:30:54.878 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T04:30:54.878 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T04:30:54.878 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T04:30:54.878 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T04:30:54.878 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T04:30:54.878 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T04:30:54.878 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T04:30:54.878 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-17T04:30:54.879 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T04:30:54.879 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T04:30:54.879 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T04:30:54.879 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T04:30:54.879 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-17T04:30:54.879 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T04:30:54.879 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T04:30:54.879 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T04:30:54.879 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T04:30:54.879 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T04:30:54.879 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T04:30:54.879 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T04:30:54.879 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T04:30:54.880 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T04:30:54.880 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T04:30:54.880 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T04:30:54.880 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-17T04:30:54.880 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T04:30:54.880 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T04:30:54.880 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T04:30:54.880 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T04:30:54.880 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-17T04:30:54.880 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-17T04:30:54.880 INFO:teuthology.orchestra.run.smithi037.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T04:30:54.880 INFO:teuthology.orchestra.run.smithi037.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T04:30:54.880 INFO:teuthology.orchestra.run.smithi037.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T04:30:54.881 INFO:teuthology.orchestra.run.smithi037.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-17T04:30:54.881 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T04:30:54.881 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-17T04:30:54.881 INFO:teuthology.orchestra.run.smithi037.stdout:Installing weak dependencies: 2024-09-17T04:30:54.881 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-17T04:30:54.881 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:30:54.881 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-09-17T04:30:54.881 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-09-17T04:30:54.881 INFO:teuthology.orchestra.run.smithi037.stdout:Install 149 Packages 2024-09-17T04:30:54.881 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrade 8 Packages 2024-09-17T04:30:54.881 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:30:54.882 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 228 M 2024-09-17T04:30:54.882 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-09-17T04:30:54.887 INFO:teuthology.orchestra.run.smithi092.stdout:(9/157): ceph-selinux-19.3.0-4989.gce1b342b.el9 609 kB/s | 25 kB 00:00 2024-09-17T04:30:55.245 INFO:teuthology.orchestra.run.smithi092.stdout:(10/157): ceph-radosgw-19.3.0-4989.gce1b342b.el 17 MB/s | 10 MB 00:00 2024-09-17T04:30:55.329 INFO:teuthology.orchestra.run.smithi092.stdout:(11/157): cephfs-mirror-19.3.0-4989.gce1b342b.e 2.6 MB/s | 225 kB 00:00 2024-09-17T04:30:55.387 INFO:teuthology.orchestra.run.smithi092.stdout:(12/157): cephfs-shell-19.3.0-4989.gce1b342b.el 465 kB/s | 27 kB 00:00 2024-09-17T04:30:55.455 INFO:teuthology.orchestra.run.smithi092.stdout:(13/157): libcephfs-devel-19.3.0-4989.gce1b342b 476 kB/s | 32 kB 00:00 2024-09-17T04:30:55.571 INFO:teuthology.orchestra.run.smithi092.stdout:(14/157): libcephfs2-19.3.0-4989.gce1b342b.el9. 6.4 MB/s | 766 kB 00:00 2024-09-17T04:30:55.581 INFO:teuthology.orchestra.run.smithi037.stdout:(1/157): ceph-19.3.0-4989.gce1b342b.el9.x86_64. 33 kB/s | 6.6 kB 00:00 2024-09-17T04:30:55.755 INFO:teuthology.orchestra.run.smithi092.stdout:(15/157): ceph-osd-19.3.0-4989.gce1b342b.el9.x8 14 MB/s | 18 MB 00:01 2024-09-17T04:30:55.780 INFO:teuthology.orchestra.run.smithi092.stdout:(16/157): libcephsqlite-19.3.0-4989.gce1b342b.e 762 kB/s | 159 kB 00:00 2024-09-17T04:30:55.823 INFO:teuthology.orchestra.run.smithi037.stdout:(2/157): ceph-fuse-19.3.0-4989.gce1b342b.el9.x8 3.5 MB/s | 873 kB 00:00 2024-09-17T04:30:55.839 INFO:teuthology.orchestra.run.smithi092.stdout:(17/157): librados-devel-19.3.0-4989.gce1b342b. 1.5 MB/s | 127 kB 00:00 2024-09-17T04:30:55.872 INFO:teuthology.orchestra.run.smithi092.stdout:(18/157): libradosstriper1-19.3.0-4989.gce1b342 5.4 MB/s | 507 kB 00:00 2024-09-17T04:30:55.923 INFO:teuthology.orchestra.run.smithi037.stdout:(3/157): ceph-immutable-object-cache-19.3.0-498 1.4 MB/s | 148 kB 00:00 2024-09-17T04:30:55.981 INFO:teuthology.orchestra.run.smithi092.stdout:(19/157): python3-ceph-argparse-19.3.0-4989.gce 416 kB/s | 45 kB 00:00 2024-09-17T04:30:55.998 INFO:teuthology.orchestra.run.smithi037.stdout:(4/157): ceph-base-19.3.0-4989.gce1b342b.el9.x8 8.9 MB/s | 5.5 MB 00:00 2024-09-17T04:30:56.065 INFO:teuthology.orchestra.run.smithi092.stdout:(20/157): python3-ceph-common-19.3.0-4989.gce1b 1.7 MB/s | 143 kB 00:00 2024-09-17T04:30:56.249 INFO:teuthology.orchestra.run.smithi037.stdout:(5/157): ceph-mgr-19.3.0-4989.gce1b342b.el9.x86 6.1 MB/s | 1.5 MB 00:00 2024-09-17T04:30:56.290 INFO:teuthology.orchestra.run.smithi092.stdout:(21/157): librgw2-19.3.0-4989.gce1b342b.el9.x86 12 MB/s | 5.3 MB 00:00 2024-09-17T04:30:56.341 INFO:teuthology.orchestra.run.smithi037.stdout:(6/157): ceph-mds-19.3.0-4989.gce1b342b.el9.x86 5.5 MB/s | 2.3 MB 00:00 2024-09-17T04:30:56.348 INFO:teuthology.orchestra.run.smithi092.stdout:(22/157): python3-rados-19.3.0-4989.gce1b342b.e 5.4 MB/s | 321 kB 00:00 2024-09-17T04:30:56.416 INFO:teuthology.orchestra.run.smithi092.stdout:(23/157): python3-cephfs-19.3.0-4989.gce1b342b. 462 kB/s | 162 kB 00:00 2024-09-17T04:30:56.441 INFO:teuthology.orchestra.run.smithi092.stdout:(24/157): python3-rbd-19.3.0-4989.gce1b342b.el9 3.2 MB/s | 305 kB 00:00 2024-09-17T04:30:56.482 INFO:teuthology.orchestra.run.smithi092.stdout:(25/157): python3-rgw-19.3.0-4989.gce1b342b.el9 1.5 MB/s | 99 kB 00:00 2024-09-17T04:30:56.508 INFO:teuthology.orchestra.run.smithi092.stdout:(26/157): rbd-fuse-19.3.0-4989.gce1b342b.el9.x8 1.3 MB/s | 86 kB 00:00 2024-09-17T04:30:56.591 INFO:teuthology.orchestra.run.smithi092.stdout:(27/157): rbd-nbd-19.3.0-4989.gce1b342b.el9.x86 2.0 MB/s | 170 kB 00:00 2024-09-17T04:30:56.667 INFO:teuthology.orchestra.run.smithi092.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4989.g 456 kB/s | 34 kB 00:00 2024-09-17T04:30:56.717 INFO:teuthology.orchestra.run.smithi092.stdout:(29/157): ceph-mgr-cephadm-19.3.0-4989.gce1b342 3.0 MB/s | 154 kB 00:00 2024-09-17T04:30:56.842 INFO:teuthology.orchestra.run.smithi092.stdout:(30/157): rbd-mirror-19.3.0-4989.gce1b342b.el9. 8.6 MB/s | 3.1 MB 00:00 2024-09-17T04:30:56.850 INFO:teuthology.orchestra.run.smithi037.stdout:(7/157): ceph-common-19.3.0-4989.gce1b342b.el9. 14 MB/s | 20 MB 00:01 2024-09-17T04:30:56.975 INFO:teuthology.orchestra.run.smithi037.stdout:(8/157): ceph-mon-19.3.0-4989.gce1b342b.el9.x86 6.8 MB/s | 5.0 MB 00:00 2024-09-17T04:30:57.075 INFO:teuthology.orchestra.run.smithi037.stdout:(9/157): ceph-selinux-19.3.0-4989.gce1b342b.el9 252 kB/s | 25 kB 00:00 2024-09-17T04:30:57.808 INFO:teuthology.orchestra.run.smithi037.stdout:(10/157): ceph-radosgw-19.3.0-4989.gce1b342b.el 11 MB/s | 10 MB 00:00 2024-09-17T04:30:57.867 INFO:teuthology.orchestra.run.smithi037.stdout:(11/157): cephfs-mirror-19.3.0-4989.gce1b342b.e 3.8 MB/s | 225 kB 00:00 2024-09-17T04:30:58.034 INFO:teuthology.orchestra.run.smithi037.stdout:(12/157): ceph-osd-19.3.0-4989.gce1b342b.el9.x8 11 MB/s | 18 MB 00:01 2024-09-17T04:30:58.059 INFO:teuthology.orchestra.run.smithi037.stdout:(13/157): cephfs-shell-19.3.0-4989.gce1b342b.el 141 kB/s | 27 kB 00:00 2024-09-17T04:30:58.059 INFO:teuthology.orchestra.run.smithi092.stdout:(31/157): ceph-test-19.3.0-4989.gce1b342b.el9.x 13 MB/s | 40 MB 00:03 2024-09-17T04:30:58.084 INFO:teuthology.orchestra.run.smithi037.stdout:(14/157): libcephfs-devel-19.3.0-4989.gce1b342b 632 kB/s | 32 kB 00:00 2024-09-17T04:30:58.143 INFO:teuthology.orchestra.run.smithi037.stdout:(15/157): libcephfs2-19.3.0-4989.gce1b342b.el9. 8.9 MB/s | 766 kB 00:00 2024-09-17T04:30:58.169 INFO:teuthology.orchestra.run.smithi037.stdout:(16/157): libcephsqlite-19.3.0-4989.gce1b342b.e 1.9 MB/s | 159 kB 00:00 2024-09-17T04:30:58.194 INFO:teuthology.orchestra.run.smithi037.stdout:(17/157): librados-devel-19.3.0-4989.gce1b342b. 2.5 MB/s | 127 kB 00:00 2024-09-17T04:30:58.210 INFO:teuthology.orchestra.run.smithi092.stdout:(32/157): ceph-mgr-dashboard-19.3.0-4989.gce1b3 9.5 MB/s | 14 MB 00:01 2024-09-17T04:30:58.227 INFO:teuthology.orchestra.run.smithi037.stdout:(18/157): libradosstriper1-19.3.0-4989.gce1b342 8.5 MB/s | 507 kB 00:00 2024-09-17T04:30:58.235 INFO:teuthology.orchestra.run.smithi092.stdout:(33/157): ceph-mgr-modules-core-19.3.0-4989.gce 1.6 MB/s | 291 kB 00:00 2024-09-17T04:30:58.260 INFO:teuthology.orchestra.run.smithi092.stdout:(34/157): ceph-mgr-rook-19.3.0-4989.gce1b342b.e 991 kB/s | 49 kB 00:00 2024-09-17T04:30:58.286 INFO:teuthology.orchestra.run.smithi092.stdout:(35/157): ceph-prometheus-alerts-19.3.0-4989.gc 327 kB/s | 16 kB 00:00 2024-09-17T04:30:58.286 INFO:teuthology.orchestra.run.smithi037.stdout:(19/157): python3-ceph-argparse-19.3.0-4989.gce 776 kB/s | 45 kB 00:00 2024-09-17T04:30:58.311 INFO:teuthology.orchestra.run.smithi092.stdout:(36/157): ceph-volume-19.3.0-4989.gce1b342b.el9 5.5 MB/s | 285 kB 00:00 2024-09-17T04:30:58.370 INFO:teuthology.orchestra.run.smithi037.stdout:(20/157): python3-ceph-common-19.3.0-4989.gce1b 1.7 MB/s | 143 kB 00:00 2024-09-17T04:30:58.394 INFO:teuthology.orchestra.run.smithi092.stdout:(37/157): ceph-mgr-diskprediction-local-19.3.0- 4.8 MB/s | 7.4 MB 00:01 2024-09-17T04:30:58.419 INFO:teuthology.orchestra.run.smithi092.stdout:(38/157): cephfs-top-19.3.0-4989.gce1b342b.el9. 168 kB/s | 18 kB 00:00 2024-09-17T04:30:58.420 INFO:teuthology.orchestra.run.smithi037.stdout:(21/157): python3-cephfs-19.3.0-4989.gce1b342b. 3.2 MB/s | 162 kB 00:00 2024-09-17T04:30:58.453 INFO:teuthology.orchestra.run.smithi092.stdout:(39/157): cephadm-19.3.0-4989.gce1b342b.el9.noa 4.6 MB/s | 784 kB 00:00 2024-09-17T04:30:58.504 INFO:teuthology.orchestra.run.smithi037.stdout:(22/157): librgw2-19.3.0-4989.gce1b342b.el9.x86 17 MB/s | 5.3 MB 00:00 2024-09-17T04:30:58.529 INFO:teuthology.orchestra.run.smithi037.stdout:(23/157): python3-rados-19.3.0-4989.gce1b342b.e 2.9 MB/s | 321 kB 00:00 2024-09-17T04:30:58.562 INFO:teuthology.orchestra.run.smithi037.stdout:(24/157): python3-rbd-19.3.0-4989.gce1b342b.el9 5.1 MB/s | 305 kB 00:00 2024-09-17T04:30:58.570 INFO:teuthology.orchestra.run.smithi092.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 455 kB/s | 80 kB 00:00 2024-09-17T04:30:58.588 INFO:teuthology.orchestra.run.smithi037.stdout:(25/157): python3-rgw-19.3.0-4989.gce1b342b.el9 1.7 MB/s | 99 kB 00:00 2024-09-17T04:30:58.613 INFO:teuthology.orchestra.run.smithi037.stdout:(26/157): rbd-fuse-19.3.0-4989.gce1b342b.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-17T04:30:58.663 INFO:teuthology.orchestra.run.smithi037.stdout:(27/157): rbd-nbd-19.3.0-4989.gce1b342b.el9.x86 3.3 MB/s | 170 kB 00:00 2024-09-17T04:30:58.695 INFO:teuthology.orchestra.run.smithi092.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 791 kB/s | 191 kB 00:00 2024-09-17T04:30:58.713 INFO:teuthology.orchestra.run.smithi037.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4989.g 685 kB/s | 34 kB 00:00 2024-09-17T04:30:58.720 INFO:teuthology.orchestra.run.smithi092.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-09-17T04:30:58.754 INFO:teuthology.orchestra.run.smithi092.stdout:(43/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.3 MB/s | 801 kB 00:00 2024-09-17T04:30:58.764 INFO:teuthology.orchestra.run.smithi037.stdout:(29/157): rbd-mirror-19.3.0-4989.gce1b342b.el9. 18 MB/s | 3.1 MB 00:00 2024-09-17T04:30:58.779 INFO:teuthology.orchestra.run.smithi092.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-17T04:30:58.789 INFO:teuthology.orchestra.run.smithi037.stdout:(30/157): ceph-mgr-cephadm-19.3.0-4989.gce1b342 2.0 MB/s | 154 kB 00:00 2024-09-17T04:30:58.836 INFO:teuthology.orchestra.run.smithi092.stdout:(45/157): python3-cryptography-36.0.1-4.el9.x86 8.7 MB/s | 1.2 MB 00:00 2024-09-17T04:30:58.863 INFO:teuthology.orchestra.run.smithi092.stdout:(46/157): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-09-17T04:30:58.888 INFO:teuthology.orchestra.run.smithi092.stdout:(47/157): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-09-17T04:30:58.914 INFO:teuthology.orchestra.run.smithi092.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 2.7 MB/s | 215 kB 00:00 2024-09-17T04:30:59.081 INFO:teuthology.orchestra.run.smithi092.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 641 kB/s | 107 kB 00:00 2024-09-17T04:30:59.139 INFO:teuthology.orchestra.run.smithi092.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 372 kB/s | 22 kB 00:00 2024-09-17T04:30:59.165 INFO:teuthology.orchestra.run.smithi092.stdout:(51/157): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-09-17T04:30:59.190 INFO:teuthology.orchestra.run.smithi092.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 943 kB 00:00 2024-09-17T04:30:59.206 INFO:teuthology.orchestra.run.smithi037.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 18 MB/s | 7.4 MB 00:00 2024-09-17T04:30:59.215 INFO:teuthology.orchestra.run.smithi092.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 650 kB/s | 33 kB 00:00 2024-09-17T04:30:59.248 INFO:teuthology.orchestra.run.smithi092.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-17T04:30:59.264 INFO:teuthology.orchestra.run.smithi037.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4989.gce 4.9 MB/s | 291 kB 00:00 2024-09-17T04:30:59.274 INFO:teuthology.orchestra.run.smithi092.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-09-17T04:30:59.308 INFO:teuthology.orchestra.run.smithi092.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-09-17T04:30:59.323 INFO:teuthology.orchestra.run.smithi037.stdout:(33/157): ceph-mgr-rook-19.3.0-4989.gce1b342b.e 845 kB/s | 49 kB 00:00 2024-09-17T04:30:59.333 INFO:teuthology.orchestra.run.smithi092.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.8 MB/s | 171 kB 00:00 2024-09-17T04:30:59.382 INFO:teuthology.orchestra.run.smithi037.stdout:(34/157): ceph-prometheus-alerts-19.3.0-4989.gc 282 kB/s | 16 kB 00:00 2024-09-17T04:30:59.391 INFO:teuthology.orchestra.run.smithi092.stdout:(58/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-09-17T04:30:59.417 INFO:teuthology.orchestra.run.smithi092.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 541 kB/s | 45 kB 00:00 2024-09-17T04:30:59.442 INFO:teuthology.orchestra.run.smithi092.stdout:(60/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-09-17T04:30:59.475 INFO:teuthology.orchestra.run.smithi092.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2024-09-17T04:30:59.501 INFO:teuthology.orchestra.run.smithi092.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 419 kB/s | 24 kB 00:00 2024-09-17T04:30:59.524 INFO:teuthology.orchestra.run.smithi037.stdout:(35/157): ceph-mgr-dashboard-19.3.0-4989.gce1b3 19 MB/s | 14 MB 00:00 2024-09-17T04:30:59.534 INFO:teuthology.orchestra.run.smithi092.stdout:(63/157): libtool-2.4.6-46.el9.x86_64.rpm 4.8 MB/s | 581 kB 00:00 2024-09-17T04:30:59.549 INFO:teuthology.orchestra.run.smithi037.stdout:(36/157): ceph-volume-19.3.0-4989.gce1b342b.el9 1.7 MB/s | 285 kB 00:00 2024-09-17T04:30:59.551 INFO:teuthology.orchestra.run.smithi092.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 627 kB/s | 47 kB 00:00 2024-09-17T04:30:59.585 INFO:teuthology.orchestra.run.smithi092.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-17T04:30:59.591 INFO:teuthology.orchestra.run.smithi037.stdout:(37/157): cephadm-19.3.0-4989.gce1b342b.el9.noa 11 MB/s | 784 kB 00:00 2024-09-17T04:30:59.616 INFO:teuthology.orchestra.run.smithi037.stdout:(38/157): cephfs-top-19.3.0-4989.gce1b342b.el9. 272 kB/s | 18 kB 00:00 2024-09-17T04:30:59.636 INFO:teuthology.orchestra.run.smithi092.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 458 kB/s | 38 kB 00:00 2024-09-17T04:30:59.661 INFO:teuthology.orchestra.run.smithi092.stdout:(67/157): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-09-17T04:30:59.733 INFO:teuthology.orchestra.run.smithi037.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 562 kB/s | 80 kB 00:00 2024-09-17T04:30:59.769 INFO:teuthology.orchestra.run.smithi092.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-17T04:30:59.791 INFO:teuthology.orchestra.run.smithi037.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-17T04:30:59.817 INFO:teuthology.orchestra.run.smithi037.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 3.9 MB/s | 801 kB 00:00 2024-09-17T04:30:59.850 INFO:teuthology.orchestra.run.smithi037.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-09-17T04:30:59.861 INFO:teuthology.orchestra.run.smithi092.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 9.7 MB/s | 910 kB 00:00 2024-09-17T04:30:59.892 INFO:teuthology.orchestra.run.smithi037.stdout:(43/157): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-09-17T04:30:59.920 INFO:teuthology.orchestra.run.smithi092.stdout:(70/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-09-17T04:30:59.926 INFO:teuthology.orchestra.run.smithi037.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-17T04:30:59.951 INFO:teuthology.orchestra.run.smithi037.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.2 MB/s | 135 kB 00:00 2024-09-17T04:30:59.977 INFO:teuthology.orchestra.run.smithi037.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-17T04:30:59.987 INFO:teuthology.orchestra.run.smithi092.stdout:(71/157): openblas-openmp-0.3.26-2.el9.x86_64.r 12 MB/s | 4.9 MB 00:00 2024-09-17T04:31:00.002 INFO:teuthology.orchestra.run.smithi037.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-17T04:31:00.012 INFO:teuthology.orchestra.run.smithi092.stdout:(72/157): postgresql-private-libs-13.16-1.el9.x 913 kB/s | 137 kB 00:00 2024-09-17T04:31:00.071 INFO:teuthology.orchestra.run.smithi092.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-09-17T04:31:00.096 INFO:teuthology.orchestra.run.smithi092.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-17T04:31:00.146 INFO:teuthology.orchestra.run.smithi092.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-17T04:31:00.221 INFO:teuthology.orchestra.run.smithi092.stdout:(76/157): postgresql-server-13.16-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-09-17T04:31:00.247 INFO:teuthology.orchestra.run.smithi092.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-17T04:31:00.269 INFO:teuthology.orchestra.run.smithi037.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-09-17T04:31:00.280 INFO:teuthology.orchestra.run.smithi092.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-17T04:31:00.306 INFO:teuthology.orchestra.run.smithi092.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 436 kB/s | 26 kB 00:00 2024-09-17T04:31:00.311 INFO:teuthology.orchestra.run.smithi037.stdout:(49/157): bison-3.7.4-5.el9.x86_64.rpm 3.0 MB/s | 943 kB 00:00 2024-09-17T04:31:00.331 INFO:teuthology.orchestra.run.smithi092.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-17T04:31:00.336 INFO:teuthology.orchestra.run.smithi037.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-09-17T04:31:00.362 INFO:teuthology.orchestra.run.smithi037.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-09-17T04:31:00.406 INFO:teuthology.orchestra.run.smithi092.stdout:(81/157): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-09-17T04:31:00.411 INFO:teuthology.orchestra.run.smithi037.stdout:(52/157): flexiblas-3.0.4-8.el9.x86_64.rpm 655 kB/s | 33 kB 00:00 2024-09-17T04:31:00.432 INFO:teuthology.orchestra.run.smithi092.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 346 kB/s | 35 kB 00:00 2024-09-17T04:31:00.437 INFO:teuthology.orchestra.run.smithi037.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 3.1 MB/s | 313 kB 00:00 2024-09-17T04:31:00.457 INFO:teuthology.orchestra.run.smithi092.stdout:(83/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-17T04:31:00.487 INFO:teuthology.orchestra.run.smithi037.stdout:(54/157): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-17T04:31:00.490 INFO:teuthology.orchestra.run.smithi092.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-17T04:31:00.515 INFO:teuthology.orchestra.run.smithi092.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-17T04:31:00.537 INFO:teuthology.orchestra.run.smithi037.stdout:(55/157): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-09-17T04:31:00.550 INFO:teuthology.orchestra.run.smithi092.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-17T04:31:00.566 INFO:teuthology.orchestra.run.smithi092.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 840 kB/s | 42 kB 00:00 2024-09-17T04:31:00.596 INFO:teuthology.orchestra.run.smithi037.stdout:(56/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.9 MB/s | 171 kB 00:00 2024-09-17T04:31:00.608 INFO:teuthology.orchestra.run.smithi092.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-17T04:31:00.633 INFO:teuthology.orchestra.run.smithi092.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-09-17T04:31:00.688 INFO:teuthology.orchestra.run.smithi037.stdout:(57/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-09-17T04:31:00.700 INFO:teuthology.orchestra.run.smithi092.stdout:(90/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-09-17T04:31:00.725 INFO:teuthology.orchestra.run.smithi092.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 555 kB/s | 51 kB 00:00 2024-09-17T04:31:00.738 INFO:teuthology.orchestra.run.smithi037.stdout:(58/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.2 MB/s | 3.0 MB 00:00 2024-09-17T04:31:00.759 INFO:teuthology.orchestra.run.smithi092.stdout:(92/157): python3-pyasn1-modules-0.4.8-6.el9.no 1.8 MB/s | 279 kB 00:00 2024-09-17T04:31:00.763 INFO:teuthology.orchestra.run.smithi037.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 602 kB/s | 45 kB 00:00 2024-09-17T04:31:00.784 INFO:teuthology.orchestra.run.smithi092.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 639 kB/s | 54 kB 00:00 2024-09-17T04:31:00.810 INFO:teuthology.orchestra.run.smithi092.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 829 kB/s | 42 kB 00:00 2024-09-17T04:31:00.831 INFO:teuthology.orchestra.run.smithi037.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 662 kB 00:00 2024-09-17T04:31:00.835 INFO:teuthology.orchestra.run.smithi092.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 877 kB/s | 44 kB 00:00 2024-09-17T04:31:00.856 INFO:teuthology.orchestra.run.smithi037.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 6.2 MB/s | 581 kB 00:00 2024-09-17T04:31:00.877 INFO:teuthology.orchestra.run.smithi092.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-09-17T04:31:00.881 INFO:teuthology.orchestra.run.smithi037.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 487 kB/s | 24 kB 00:00 2024-09-17T04:31:00.903 INFO:teuthology.orchestra.run.smithi092.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 535 kB/s | 36 kB 00:00 2024-09-17T04:31:00.906 INFO:teuthology.orchestra.run.smithi037.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-17T04:31:00.927 INFO:teuthology.orchestra.run.smithi092.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-17T04:31:00.961 INFO:teuthology.orchestra.run.smithi092.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-17T04:31:00.986 INFO:teuthology.orchestra.run.smithi092.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 985 kB/s | 58 kB 00:00 2024-09-17T04:31:01.020 INFO:teuthology.orchestra.run.smithi092.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-17T04:31:01.046 INFO:teuthology.orchestra.run.smithi092.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-17T04:31:01.071 INFO:teuthology.orchestra.run.smithi092.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 890 kB/s | 45 kB 00:00 2024-09-17T04:31:01.162 INFO:teuthology.orchestra.run.smithi092.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 191 kB/s | 22 kB 00:00 2024-09-17T04:31:01.221 INFO:teuthology.orchestra.run.smithi092.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-17T04:31:01.248 INFO:teuthology.orchestra.run.smithi037.stdout:(64/157): ceph-test-19.3.0-4989.gce1b342b.el9.x 9.6 MB/s | 40 MB 00:04 2024-09-17T04:31:01.263 INFO:teuthology.orchestra.run.smithi092.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 7.3 MB/s | 308 kB 00:00 2024-09-17T04:31:01.274 INFO:teuthology.orchestra.run.smithi037.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 513 kB/s | 188 kB 00:00 2024-09-17T04:31:01.288 INFO:teuthology.orchestra.run.smithi092.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 776 kB/s | 19 kB 00:00 2024-09-17T04:31:01.299 INFO:teuthology.orchestra.run.smithi037.stdout:(66/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 701 kB/s | 292 kB 00:00 2024-09-17T04:31:01.383 INFO:teuthology.orchestra.run.smithi037.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 288 kB/s | 38 kB 00:00 2024-09-17T04:31:01.455 INFO:teuthology.orchestra.run.smithi092.stdout:(108/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-09-17T04:31:01.506 INFO:teuthology.orchestra.run.smithi092.stdout:(109/157): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-09-17T04:31:01.539 INFO:teuthology.orchestra.run.smithi092.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 312 kB/s | 26 kB 00:00 2024-09-17T04:31:01.573 INFO:teuthology.orchestra.run.smithi092.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.1 MB/s | 67 kB 00:00 2024-09-17T04:31:01.591 INFO:teuthology.orchestra.run.smithi037.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-09-17T04:31:01.598 INFO:teuthology.orchestra.run.smithi092.stdout:(112/157): liboath-2.6.7-2.el9.x86_64.rpm 537 kB/s | 49 kB 00:00 2024-09-17T04:31:01.623 INFO:teuthology.orchestra.run.smithi092.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-17T04:31:01.674 INFO:teuthology.orchestra.run.smithi037.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 11 MB/s | 910 kB 00:00 2024-09-17T04:31:01.699 INFO:teuthology.orchestra.run.smithi092.stdout:(114/157): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-17T04:31:01.724 INFO:teuthology.orchestra.run.smithi092.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-17T04:31:01.733 INFO:teuthology.orchestra.run.smithi037.stdout:(70/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-09-17T04:31:01.757 INFO:teuthology.orchestra.run.smithi092.stdout:(116/157): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-09-17T04:31:01.758 INFO:teuthology.orchestra.run.smithi037.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.6 MB/s | 137 kB 00:00 2024-09-17T04:31:01.783 INFO:teuthology.orchestra.run.smithi092.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 550 kB/s | 32 kB 00:00 2024-09-17T04:31:01.808 INFO:teuthology.orchestra.run.smithi092.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-17T04:31:01.825 INFO:teuthology.orchestra.run.smithi037.stdout:(72/157): openblas-openmp-0.3.26-2.el9.x86_64.r 9.0 MB/s | 4.9 MB 00:00 2024-09-17T04:31:01.841 INFO:teuthology.orchestra.run.smithi092.stdout:(119/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.4 MB/s | 839 kB 00:00 2024-09-17T04:31:01.867 INFO:teuthology.orchestra.run.smithi092.stdout:(120/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-09-17T04:31:01.892 INFO:teuthology.orchestra.run.smithi037.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 7.6 MB/s | 1.0 MB 00:00 2024-09-17T04:31:01.893 INFO:teuthology.orchestra.run.smithi092.stdout:(121/157): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-17T04:31:01.917 INFO:teuthology.orchestra.run.smithi037.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 955 kB/s | 87 kB 00:00 2024-09-17T04:31:01.918 INFO:teuthology.orchestra.run.smithi092.stdout:(122/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-09-17T04:31:01.943 INFO:teuthology.orchestra.run.smithi092.stdout:(123/157): python3-google-auth-2.34.0-1.el9.noa 2.9 MB/s | 220 kB 00:00 2024-09-17T04:31:01.976 INFO:teuthology.orchestra.run.smithi037.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-17T04:31:01.976 INFO:teuthology.orchestra.run.smithi092.stdout:(124/157): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-09-17T04:31:02.002 INFO:teuthology.orchestra.run.smithi092.stdout:(125/157): python3-jaraco-8.2.1-3.el9.noarch.rp 181 kB/s | 11 kB 00:00 2024-09-17T04:31:02.027 INFO:teuthology.orchestra.run.smithi092.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 350 kB/s | 18 kB 00:00 2024-09-17T04:31:02.052 INFO:teuthology.orchestra.run.smithi092.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-09-17T04:31:02.078 INFO:teuthology.orchestra.run.smithi092.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-09-17T04:31:02.079 INFO:teuthology.orchestra.run.smithi037.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.3 MB/s | 245 kB 00:00 2024-09-17T04:31:02.095 INFO:teuthology.orchestra.run.smithi092.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 469 kB/s | 20 kB 00:00 2024-09-17T04:31:02.120 INFO:teuthology.orchestra.run.smithi092.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-17T04:31:02.135 INFO:teuthology.orchestra.run.smithi037.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.3 MB/s | 248 kB 00:00 2024-09-17T04:31:02.153 INFO:teuthology.orchestra.run.smithi092.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-17T04:31:02.187 INFO:teuthology.orchestra.run.smithi092.stdout:(132/157): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-09-17T04:31:02.210 INFO:teuthology.orchestra.run.smithi037.stdout:(78/157): postgresql-server-13.16-1.el9.x86_64. 12 MB/s | 5.8 MB 00:00 2024-09-17T04:31:02.212 INFO:teuthology.orchestra.run.smithi092.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 987 kB/s | 58 kB 00:00 2024-09-17T04:31:02.235 INFO:teuthology.orchestra.run.smithi037.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-09-17T04:31:02.237 INFO:teuthology.orchestra.run.smithi092.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 661 kB/s | 16 kB 00:00 2024-09-17T04:31:02.271 INFO:teuthology.orchestra.run.smithi092.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-17T04:31:02.311 INFO:teuthology.orchestra.run.smithi037.stdout:(80/157): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-09-17T04:31:02.313 INFO:teuthology.orchestra.run.smithi092.stdout:(136/157): python3-grpcio-1.46.7-10.el9.x86_64. 4.8 MB/s | 2.0 MB 00:00 2024-09-17T04:31:02.336 INFO:teuthology.orchestra.run.smithi037.stdout:(81/157): python3-jsonpointer-2.0-4.el9.noarch. 152 kB/s | 19 kB 00:00 2024-09-17T04:31:02.354 INFO:teuthology.orchestra.run.smithi092.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 772 kB/s | 90 kB 00:00 2024-09-17T04:31:02.361 INFO:teuthology.orchestra.run.smithi037.stdout:(82/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-17T04:31:02.380 INFO:teuthology.orchestra.run.smithi092.stdout:(138/157): python3-pyperclip-1.8.2-2.el9.noarch 259 kB/s | 28 kB 00:00 2024-09-17T04:31:02.387 INFO:teuthology.orchestra.run.smithi037.stdout:(83/157): python3-markupsafe-1.1.1-12.el9.x86_6 460 kB/s | 35 kB 00:00 2024-09-17T04:31:02.406 INFO:teuthology.orchestra.run.smithi092.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 335 kB/s | 31 kB 00:00 2024-09-17T04:31:02.420 INFO:teuthology.orchestra.run.smithi037.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-17T04:31:02.431 INFO:teuthology.orchestra.run.smithi092.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-17T04:31:02.445 INFO:teuthology.orchestra.run.smithi037.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-17T04:31:02.456 INFO:teuthology.orchestra.run.smithi092.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 715 kB/s | 36 kB 00:00 2024-09-17T04:31:02.479 INFO:teuthology.orchestra.run.smithi037.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-17T04:31:02.481 INFO:teuthology.orchestra.run.smithi092.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-17T04:31:02.504 INFO:teuthology.orchestra.run.smithi037.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 720 kB/s | 42 kB 00:00 2024-09-17T04:31:02.507 INFO:teuthology.orchestra.run.smithi092.stdout:(143/157): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-17T04:31:02.532 INFO:teuthology.orchestra.run.smithi092.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-17T04:31:02.538 INFO:teuthology.orchestra.run.smithi037.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-17T04:31:02.563 INFO:teuthology.orchestra.run.smithi037.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-17T04:31:02.565 INFO:teuthology.orchestra.run.smithi092.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 601 kB/s | 20 kB 00:00 2024-09-17T04:31:02.591 INFO:teuthology.orchestra.run.smithi092.stdout:(146/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-09-17T04:31:02.616 INFO:teuthology.orchestra.run.smithi092.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-17T04:31:02.638 INFO:teuthology.orchestra.run.smithi037.stdout:(90/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-09-17T04:31:02.641 INFO:teuthology.orchestra.run.smithi092.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-17T04:31:02.663 INFO:teuthology.orchestra.run.smithi037.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 508 kB/s | 51 kB 00:00 2024-09-17T04:31:02.689 INFO:teuthology.orchestra.run.smithi037.stdout:(92/157): python3-pyasn1-modules-0.4.8-6.el9.no 1.8 MB/s | 279 kB 00:00 2024-09-17T04:31:02.708 INFO:teuthology.orchestra.run.smithi092.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-17T04:31:02.714 INFO:teuthology.orchestra.run.smithi037.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 710 kB/s | 54 kB 00:00 2024-09-17T04:31:02.739 INFO:teuthology.orchestra.run.smithi037.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 832 kB/s | 42 kB 00:00 2024-09-17T04:31:02.773 INFO:teuthology.orchestra.run.smithi037.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 756 kB/s | 44 kB 00:00 2024-09-17T04:31:02.798 INFO:teuthology.orchestra.run.smithi037.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-17T04:31:02.823 INFO:teuthology.orchestra.run.smithi037.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 714 kB/s | 36 kB 00:00 2024-09-17T04:31:02.840 INFO:teuthology.orchestra.run.smithi037.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-17T04:31:02.892 INFO:teuthology.orchestra.run.smithi037.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 4.4 MB/s | 304 kB 00:00 2024-09-17T04:31:02.900 INFO:teuthology.orchestra.run.smithi092.stdout:(150/157): NetworkManager-1.51.0-1.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2024-09-17T04:31:02.907 INFO:teuthology.orchestra.run.smithi037.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 863 kB/s | 58 kB 00:00 2024-09-17T04:31:02.958 INFO:teuthology.orchestra.run.smithi037.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.4 MB/s | 96 kB 00:00 2024-09-17T04:31:02.958 INFO:teuthology.orchestra.run.smithi092.stdout:(151/157): librados2-19.3.0-4989.gce1b342b.el9. 9.9 MB/s | 3.4 MB 00:00 2024-09-17T04:31:02.975 INFO:teuthology.orchestra.run.smithi037.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 950 kB/s | 64 kB 00:00 2024-09-17T04:31:03.000 INFO:teuthology.orchestra.run.smithi037.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-09-17T04:31:03.000 INFO:teuthology.orchestra.run.smithi092.stdout:(152/157): NetworkManager-libnm-1.51.0-1.el9.x8 18 MB/s | 1.8 MB 00:00 2024-09-17T04:31:03.034 INFO:teuthology.orchestra.run.smithi092.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 544 kB/s | 41 kB 00:00 2024-09-17T04:31:03.076 INFO:teuthology.orchestra.run.smithi092.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 3.2 MB/s | 249 kB 00:00 2024-09-17T04:31:03.126 INFO:teuthology.orchestra.run.smithi092.stdout:(155/157): librbd1-19.3.0-4989.gce1b342b.el9.x8 6.5 MB/s | 3.2 MB 00:00 2024-09-17T04:31:03.185 INFO:teuthology.orchestra.run.smithi092.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-09-17T04:31:03.277 INFO:teuthology.orchestra.run.smithi092.stdout:(157/157): openssl-3.2.2-6.el9.x86_64.rpm 5.6 MB/s | 1.4 MB 00:00 2024-09-17T04:31:03.283 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-17T04:31:03.283 INFO:teuthology.orchestra.run.smithi092.stdout:Total 22 MB/s | 228 MB 00:10 2024-09-17T04:31:03.292 INFO:teuthology.orchestra.run.smithi037.stdout:(104/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-09-17T04:31:03.309 INFO:teuthology.orchestra.run.smithi037.stdout:(105/157): lua-devel-5.4.4-4.el9.x86_64.rpm 67 kB/s | 22 kB 00:00 2024-09-17T04:31:03.384 INFO:teuthology.orchestra.run.smithi037.stdout:(106/157): abseil-cpp-20211102.0-3.el9.x86_64.r 5.8 MB/s | 548 kB 00:00 2024-09-17T04:31:03.401 INFO:teuthology.orchestra.run.smithi037.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 1.1 MB/s | 19 kB 00:00 2024-09-17T04:31:03.443 INFO:teuthology.orchestra.run.smithi037.stdout:(108/157): gperftools-libs-2.9.1-3.el9.x86_64.r 2.2 MB/s | 308 kB 00:00 2024-09-17T04:31:03.468 INFO:teuthology.orchestra.run.smithi037.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-17T04:31:03.494 INFO:teuthology.orchestra.run.smithi037.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-17T04:31:03.527 INFO:teuthology.orchestra.run.smithi037.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-17T04:31:03.561 INFO:teuthology.orchestra.run.smithi037.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-17T04:31:03.645 INFO:teuthology.orchestra.run.smithi037.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 839 kB 00:00 2024-09-17T04:31:03.712 INFO:teuthology.orchestra.run.smithi037.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-09-17T04:31:03.737 INFO:teuthology.orchestra.run.smithi037.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-17T04:31:03.762 INFO:teuthology.orchestra.run.smithi037.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-17T04:31:03.954 INFO:teuthology.orchestra.run.smithi037.stdout:(117/157): libarrow-9.0.0-11.el9.x86_64.rpm 8.0 MB/s | 4.4 MB 00:00 2024-09-17T04:31:03.979 INFO:teuthology.orchestra.run.smithi037.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 65 kB/s | 14 kB 00:00 2024-09-17T04:31:04.005 INFO:teuthology.orchestra.run.smithi037.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-17T04:31:04.038 INFO:teuthology.orchestra.run.smithi037.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-17T04:31:04.063 INFO:teuthology.orchestra.run.smithi037.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-17T04:31:04.089 INFO:teuthology.orchestra.run.smithi037.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-17T04:31:04.122 INFO:teuthology.orchestra.run.smithi037.stdout:(123/157): protobuf-compiler-3.14.0-13.el9.x86_ 769 kB/s | 863 kB 00:01 2024-09-17T04:31:04.139 INFO:teuthology.orchestra.run.smithi037.stdout:(124/157): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-17T04:31:04.173 INFO:teuthology.orchestra.run.smithi037.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 685 kB/s | 18 kB 00:00 2024-09-17T04:31:04.198 INFO:teuthology.orchestra.run.smithi037.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 1.0 MB/s | 23 kB 00:00 2024-09-17T04:31:04.223 INFO:teuthology.orchestra.run.smithi037.stdout:(127/157): python3-jaraco-8.2.1-3.el9.noarch.rp 106 kB/s | 11 kB 00:00 2024-09-17T04:31:04.248 INFO:teuthology.orchestra.run.smithi037.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-17T04:31:04.274 INFO:teuthology.orchestra.run.smithi037.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-09-17T04:31:04.307 INFO:teuthology.orchestra.run.smithi037.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-17T04:31:04.325 INFO:teuthology.orchestra.run.smithi037.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 4.7 MB/s | 79 kB 00:00 2024-09-17T04:31:04.375 INFO:teuthology.orchestra.run.smithi037.stdout:(132/157): python3-grpcio-1.46.7-10.el9.x86_64. 6.6 MB/s | 2.0 MB 00:00 2024-09-17T04:31:04.400 INFO:teuthology.orchestra.run.smithi037.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 766 kB/s | 58 kB 00:00 2024-09-17T04:31:04.434 INFO:teuthology.orchestra.run.smithi037.stdout:(134/157): python3-kubernetes-26.1.0-2.el9.noar 5.5 MB/s | 1.0 MB 00:00 2024-09-17T04:31:04.458 INFO:teuthology.orchestra.run.smithi037.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-09-17T04:31:04.484 INFO:teuthology.orchestra.run.smithi037.stdout:(136/157): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-09-17T04:31:04.504 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-17T04:31:04.509 INFO:teuthology.orchestra.run.smithi037.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-17T04:31:04.534 INFO:teuthology.orchestra.run.smithi037.stdout:(138/157): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-09-17T04:31:04.560 INFO:teuthology.orchestra.run.smithi037.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-09-17T04:31:04.585 INFO:teuthology.orchestra.run.smithi037.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-17T04:31:04.610 INFO:teuthology.orchestra.run.smithi037.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-09-17T04:31:04.636 INFO:teuthology.orchestra.run.smithi037.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-17T04:31:04.657 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-17T04:31:04.657 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-17T04:31:04.661 INFO:teuthology.orchestra.run.smithi037.stdout:(143/157): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-17T04:31:04.686 INFO:teuthology.orchestra.run.smithi037.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-17T04:31:04.711 INFO:teuthology.orchestra.run.smithi037.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-17T04:31:04.728 INFO:teuthology.orchestra.run.smithi037.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 477 kB/s | 20 kB 00:00 2024-09-17T04:31:04.753 INFO:teuthology.orchestra.run.smithi037.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-17T04:31:04.779 INFO:teuthology.orchestra.run.smithi037.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-09-17T04:31:04.879 INFO:teuthology.orchestra.run.smithi037.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-17T04:31:05.046 INFO:teuthology.orchestra.run.smithi037.stdout:(150/157): librbd1-19.3.0-4989.gce1b342b.el9.x8 12 MB/s | 3.2 MB 00:00 2024-09-17T04:31:05.096 INFO:teuthology.orchestra.run.smithi037.stdout:(151/157): NetworkManager-1.51.0-1.el9.x86_64.r 11 MB/s | 2.3 MB 00:00 2024-09-17T04:31:05.138 INFO:teuthology.orchestra.run.smithi037.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 982 kB/s | 41 kB 00:00 2024-09-17T04:31:05.180 INFO:teuthology.orchestra.run.smithi037.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 5.9 MB/s | 249 kB 00:00 2024-09-17T04:31:05.230 INFO:teuthology.orchestra.run.smithi037.stdout:(154/157): librados2-19.3.0-4989.gce1b342b.el9. 7.1 MB/s | 3.4 MB 00:00 2024-09-17T04:31:05.272 INFO:teuthology.orchestra.run.smithi037.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 15 MB/s | 1.4 MB 00:00 2024-09-17T04:31:05.564 INFO:teuthology.orchestra.run.smithi037.stdout:(156/157): NetworkManager-libnm-1.51.0-1.el9.x8 3.5 MB/s | 1.8 MB 00:00 2024-09-17T04:31:05.839 INFO:teuthology.orchestra.run.smithi037.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.9 MB/s | 2.4 MB 00:00 2024-09-17T04:31:05.845 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-09-17T04:31:05.845 INFO:teuthology.orchestra.run.smithi037.stdout:Total 21 MB/s | 228 MB 00:10 2024-09-17T04:31:06.916 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-17T04:31:06.917 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-17T04:31:07.083 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-09-17T04:31:07.254 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-09-17T04:31:07.254 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-09-17T04:31:09.764 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-09-17T04:31:09.765 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-09-17T04:31:13.352 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-17T04:31:13.497 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-17T04:31:13.536 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-17T04:31:13.589 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-17T04:31:13.622 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-17T04:31:13.647 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-17T04:31:13.701 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-17T04:31:14.104 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-17T04:31:14.173 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librados2-2:19.3.0-4989.gce1b342b.el9.x86_64 8/165 2024-09-17T04:31:14.283 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:19.3.0-4989.gce1b342b.el9.x86_64 8/165 2024-09-17T04:31:14.316 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs2-2:19.3.0-4989.gce1b342b.el9.x86_64 9/165 2024-09-17T04:31:14.408 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs2-2:19.3.0-4989.gce1b342b.el9.x86_64 9/165 2024-09-17T04:31:14.437 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rados-2:19.3.0-4989.gce1b342b.el9.x86_64 10/165 2024-09-17T04:31:14.467 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-17T04:31:14.639 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-17T04:31:14.694 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-17T04:31:14.729 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephsqlite-2:19.3.0-4989.gce1b342b.el9.x86_64 14/165 2024-09-17T04:31:14.835 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4989.gce1b342b.el9.x86_64 14/165 2024-09-17T04:31:14.863 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libradosstriper1-2:19.3.0-4989.gce1b342b.el9.x86 15/165 2024-09-17T04:31:14.927 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4989.gce1b342b.el9.x86 15/165 2024-09-17T04:31:14.948 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-17T04:31:14.972 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-17T04:31:15.137 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-17T04:31:15.315 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-17T04:31:15.425 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-17T04:31:15.535 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-17T04:31:15.631 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-17T04:31:15.868 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-17T04:31:15.980 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-17T04:31:16.019 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-17T04:31:16.061 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-17T04:31:16.143 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-17T04:31:16.188 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-17T04:31:16.212 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-09-17T04:31:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-17T04:31:16.271 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-17T04:31:16.305 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-17T04:31:16.332 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-17T04:31:16.344 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-17T04:31:16.401 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-17T04:31:16.441 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-17T04:31:16.478 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-17T04:31:16.518 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-17T04:31:16.561 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-17T04:31:16.617 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-17T04:31:16.637 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-17T04:31:16.654 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-17T04:31:16.672 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-common-2:19.3.0-4989.gce1b342b.el9. 34/165 2024-09-17T04:31:16.678 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-17T04:31:16.715 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-argparse-2:19.3.0-4989.gce1b342b.el 35/165 2024-09-17T04:31:16.734 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-17T04:31:16.744 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cephfs-2:19.3.0-4989.gce1b342b.el9.x86_6 36/165 2024-09-17T04:31:16.814 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-17T04:31:16.858 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-17T04:31:16.933 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-17T04:31:17.074 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-17T04:31:17.090 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-17T04:31:17.136 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-17T04:31:17.212 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librados2-2:19.3.0-4989.gce1b342b.el9.x86_64 8/165 2024-09-17T04:31:17.341 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:19.3.0-4989.gce1b342b.el9.x86_64 8/165 2024-09-17T04:31:17.379 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs2-2:19.3.0-4989.gce1b342b.el9.x86_64 9/165 2024-09-17T04:31:17.472 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephfs2-2:19.3.0-4989.gce1b342b.el9.x86_64 9/165 2024-09-17T04:31:17.509 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rados-2:19.3.0-4989.gce1b342b.el9.x86_64 10/165 2024-09-17T04:31:17.539 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-17T04:31:17.661 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-17T04:31:17.709 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-17T04:31:17.740 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-17T04:31:17.774 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-17T04:31:17.777 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-17T04:31:17.800 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephsqlite-2:19.3.0-4989.gce1b342b.el9.x86_64 14/165 2024-09-17T04:31:17.821 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-17T04:31:17.858 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librados-devel-2:19.3.0-4989.gce1b342b.el9.x86_6 46/165 2024-09-17T04:31:17.891 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4989.gce1b342b.el9.x86_64 14/165 2024-09-17T04:31:17.909 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-17T04:31:17.918 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libradosstriper1-2:19.3.0-4989.gce1b342b.el9.x86 15/165 2024-09-17T04:31:17.965 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4989.gce1b342b.el9.x86 15/165 2024-09-17T04:31:17.994 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-17T04:31:18.018 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-17T04:31:18.067 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-17T04:31:18.128 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-17T04:31:18.219 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-17T04:31:18.230 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-17T04:31:18.284 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-17T04:31:18.356 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-17T04:31:18.388 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-17T04:31:18.414 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-17T04:31:18.424 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-17T04:31:18.470 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-17T04:31:18.506 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-17T04:31:18.525 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-17T04:31:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-17T04:31:18.631 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-17T04:31:18.702 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-17T04:31:18.922 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-17T04:31:18.945 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-17T04:31:19.060 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-17T04:31:19.066 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-17T04:31:19.098 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-17T04:31:19.110 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-17T04:31:19.133 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-17T04:31:19.152 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-17T04:31:19.209 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-17T04:31:19.255 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-17T04:31:19.260 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-17T04:31:19.294 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-17T04:31:19.300 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-17T04:31:19.344 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-17T04:31:19.351 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-17T04:31:19.379 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-17T04:31:19.384 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-17T04:31:19.404 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-17T04:31:19.411 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-17T04:31:19.483 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-17T04:31:19.504 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-17T04:31:19.535 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-17T04:31:19.564 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-17T04:31:19.617 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-17T04:31:19.628 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-17T04:31:19.669 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-17T04:31:19.696 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-17T04:31:19.705 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-17T04:31:19.727 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-common-2:19.3.0-4989.gce1b342b.el9. 34/165 2024-09-17T04:31:19.740 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-17T04:31:19.771 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-argparse-2:19.3.0-4989.gce1b342b.el 35/165 2024-09-17T04:31:19.778 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-17T04:31:19.807 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cephfs-2:19.3.0-4989.gce1b342b.el9.x86_6 36/165 2024-09-17T04:31:19.810 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-17T04:31:19.840 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-17T04:31:19.875 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-17T04:31:19.880 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-17T04:31:19.908 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-17T04:31:19.931 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-17T04:31:19.975 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-17T04:31:20.009 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-17T04:31:20.009 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-17T04:31:20.095 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-17T04:31:20.139 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-17T04:31:20.154 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-17T04:31:20.171 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-17T04:31:20.209 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-17T04:31:20.274 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-17T04:31:20.330 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-17T04:31:20.765 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-17T04:31:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-17T04:31:20.882 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-17T04:31:20.926 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-17T04:31:20.963 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librados-devel-2:19.3.0-4989.gce1b342b.el9.x86_6 46/165 2024-09-17T04:31:21.006 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-17T04:31:21.050 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-17T04:31:21.110 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-17T04:31:21.144 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-17T04:31:21.185 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-17T04:31:21.242 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-17T04:31:21.249 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-17T04:31:21.299 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-17T04:31:21.343 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-17T04:31:21.344 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-17T04:31:21.407 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-17T04:31:21.431 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-17T04:31:21.471 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-17T04:31:21.481 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-17T04:31:21.517 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-17T04:31:21.562 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-17T04:31:21.618 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-17T04:31:21.664 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-17T04:31:21.702 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-17T04:31:21.780 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-17T04:31:21.812 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-17T04:31:21.856 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-17T04:31:21.899 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-17T04:31:22.043 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-17T04:31:22.082 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-17T04:31:22.124 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-17T04:31:22.231 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-17T04:31:22.274 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-17T04:31:22.315 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-17T04:31:22.347 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-17T04:31:22.371 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-17T04:31:22.445 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-17T04:31:22.477 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-17T04:31:22.558 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-17T04:31:22.613 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-17T04:31:22.662 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-17T04:31:22.690 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-17T04:31:22.724 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-17T04:31:22.764 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-17T04:31:22.804 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-17T04:31:22.834 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-17T04:31:22.868 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-17T04:31:22.901 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-17T04:31:22.943 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-17T04:31:22.980 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-17T04:31:23.070 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-17T04:31:23.119 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-17T04:31:23.159 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-17T04:31:23.267 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-17T04:31:23.327 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-17T04:31:23.827 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-17T04:31:23.914 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-17T04:31:23.941 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-17T04:31:23.975 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-17T04:31:24.098 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-17T04:31:24.148 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-17T04:31:24.181 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-17T04:31:24.294 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-17T04:31:24.300 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-17T04:31:24.346 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librbd1-2:19.3.0-4989.gce1b342b.el9.x86_64 102/165 2024-09-17T04:31:24.368 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-17T04:31:24.417 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-17T04:31:24.437 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:19.3.0-4989.gce1b342b.el9.x86_64 102/165 2024-09-17T04:31:24.467 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rbd-2:19.3.0-4989.gce1b342b.el9.x86_64 103/165 2024-09-17T04:31:24.506 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-17T04:31:24.512 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-17T04:31:24.543 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-17T04:31:24.736 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-17T04:31:24.757 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-17T04:31:24.845 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-17T04:31:25.021 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-17T04:31:25.057 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librgw2-2:19.3.0-4989.gce1b342b.el9.x86_64 106/165 2024-09-17T04:31:25.128 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librgw2-2:19.3.0-4989.gce1b342b.el9.x86_64 106/165 2024-09-17T04:31:25.150 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rgw-2:19.3.0-4989.gce1b342b.el9.x86_64 107/165 2024-09-17T04:31:25.542 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-17T04:31:25.666 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-17T04:31:27.002 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-17T04:31:27.076 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-17T04:31:27.104 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-17T04:31:27.137 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-17T04:31:27.453 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 108/165 2024-09-17T04:31:27.481 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-17T04:31:27.490 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 108/165 2024-09-17T04:31:27.525 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librbd1-2:19.3.0-4989.gce1b342b.el9.x86_64 102/165 2024-09-17T04:31:27.600 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:19.3.0-4989.gce1b342b.el9.x86_64 102/165 2024-09-17T04:31:27.630 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rbd-2:19.3.0-4989.gce1b342b.el9.x86_64 103/165 2024-09-17T04:31:27.666 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-17T04:31:28.062 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 108/165 2024-09-17T04:31:28.101 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-base-2:19.3.0-4989.gce1b342b.el9.x86_64 109/165 2024-09-17T04:31:28.195 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-17T04:31:28.198 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-base-2:19.3.0-4989.gce1b342b.el9.x86_64 109/165 2024-09-17T04:31:28.198 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T04:31:28.198 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T04:31:28.198 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:31:28.219 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librgw2-2:19.3.0-4989.gce1b342b.el9.x86_64 106/165 2024-09-17T04:31:28.236 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-selinux-2:19.3.0-4989.gce1b342b.el9.x86_64 110/165 2024-09-17T04:31:28.282 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librgw2-2:19.3.0-4989.gce1b342b.el9.x86_64 106/165 2024-09-17T04:31:28.313 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rgw-2:19.3.0-4989.gce1b342b.el9.x86_64 107/165 2024-09-17T04:31:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 108/165 2024-09-17T04:31:30.761 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 108/165 2024-09-17T04:31:31.365 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 108/165 2024-09-17T04:31:31.405 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-base-2:19.3.0-4989.gce1b342b.el9.x86_64 109/165 2024-09-17T04:31:31.498 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-base-2:19.3.0-4989.gce1b342b.el9.x86_64 109/165 2024-09-17T04:31:31.499 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T04:31:31.499 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T04:31:31.499 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:31:31.540 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-selinux-2:19.3.0-4989.gce1b342b.el9.x86_64 110/165 2024-09-17T04:31:40.354 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4989.gce1b342b.el9.x86_64 110/165 2024-09-17T04:31:40.354 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /sys 2024-09-17T04:31:40.355 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /proc 2024-09-17T04:31:40.355 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /mnt 2024-09-17T04:31:40.355 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /var/tmp 2024-09-17T04:31:40.355 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /home 2024-09-17T04:31:40.355 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /root 2024-09-17T04:31:40.355 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /tmp 2024-09-17T04:31:40.355 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:31:40.529 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-volume-2:19.3.0-4989.gce1b342b.el9.noarch 111/165 2024-09-17T04:31:40.545 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-volume-2:19.3.0-4989.gce1b342b.el9.noarch 111/165 2024-09-17T04:31:40.545 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:31:40.545 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T04:31:40.545 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:31:42.132 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-osd-2:19.3.0-4989.gce1b342b.el9.x86_64 112/165 2024-09-17T04:31:42.158 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-osd-2:19.3.0-4989.gce1b342b.el9.x86_64 112/165 2024-09-17T04:31:42.158 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:31:42.158 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T04:31:42.158 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T04:31:42.158 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T04:31:42.158 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:31:42.399 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mds-2:19.3.0-4989.gce1b342b.el9.x86_64 113/165 2024-09-17T04:31:42.421 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mds-2:19.3.0-4989.gce1b342b.el9.x86_64 113/165 2024-09-17T04:31:42.422 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:31:42.422 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T04:31:42.422 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T04:31:42.422 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T04:31:42.422 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:31:42.907 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mon-2:19.3.0-4989.gce1b342b.el9.x86_64 114/165 2024-09-17T04:31:42.931 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mon-2:19.3.0-4989.gce1b342b.el9.x86_64 114/165 2024-09-17T04:31:42.931 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:31:42.931 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T04:31:42.931 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T04:31:42.931 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T04:31:42.932 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:31:43.038 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-17T04:31:43.095 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-17T04:31:43.150 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-17T04:31:43.335 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-17T04:31:43.383 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-17T04:31:43.461 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-17T04:31:43.549 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-17T04:31:43.689 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-17T04:31:43.734 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-17T04:31:43.765 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-17T04:31:43.793 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-17T04:31:44.239 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 126/165 2024-09-17T04:31:44.266 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 126/165 2024-09-17T04:31:44.298 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4989.gce1b342b.e 127/165 2024-09-17T04:31:44.365 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4989.gce1b342b. 128/165 2024-09-17T04:31:44.389 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4989.gce1b342b.el9.noa 129/165 2024-09-17T04:31:44.390 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4989.gce1b342b.el9.x86_64 110/165 2024-09-17T04:31:44.390 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /sys 2024-09-17T04:31:44.391 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /proc 2024-09-17T04:31:44.391 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /mnt 2024-09-17T04:31:44.391 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /var/tmp 2024-09-17T04:31:44.391 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /home 2024-09-17T04:31:44.391 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /root 2024-09-17T04:31:44.391 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /tmp 2024-09-17T04:31:44.391 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:31:44.567 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-volume-2:19.3.0-4989.gce1b342b.el9.noarch 111/165 2024-09-17T04:31:44.584 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-volume-2:19.3.0-4989.gce1b342b.el9.noarch 111/165 2024-09-17T04:31:44.584 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:31:44.585 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T04:31:44.585 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:31:46.245 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-osd-2:19.3.0-4989.gce1b342b.el9.x86_64 112/165 2024-09-17T04:31:46.273 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-osd-2:19.3.0-4989.gce1b342b.el9.x86_64 112/165 2024-09-17T04:31:46.273 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:31:46.273 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T04:31:46.273 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T04:31:46.274 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T04:31:46.274 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:31:46.512 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mds-2:19.3.0-4989.gce1b342b.el9.x86_64 113/165 2024-09-17T04:31:46.537 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mds-2:19.3.0-4989.gce1b342b.el9.x86_64 113/165 2024-09-17T04:31:46.537 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:31:46.537 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T04:31:46.537 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T04:31:46.537 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T04:31:46.537 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:31:47.038 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mon-2:19.3.0-4989.gce1b342b.el9.x86_64 114/165 2024-09-17T04:31:47.062 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mon-2:19.3.0-4989.gce1b342b.el9.x86_64 114/165 2024-09-17T04:31:47.063 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:31:47.063 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T04:31:47.063 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T04:31:47.063 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T04:31:47.063 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:31:47.177 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-17T04:31:47.235 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-17T04:31:47.299 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-17T04:31:47.501 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-17T04:31:47.556 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-17T04:31:47.567 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4989.gce1b342b.el9.noa 129/165 2024-09-17T04:31:47.629 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4989.gce1b342b.el9.n 130/165 2024-09-17T04:31:47.635 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-17T04:31:47.733 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-17T04:31:47.885 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-17T04:31:47.931 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-17T04:31:47.963 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-17T04:31:47.990 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-17T04:31:48.446 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 126/165 2024-09-17T04:31:48.472 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 126/165 2024-09-17T04:31:48.504 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4989.gce1b342b.e 127/165 2024-09-17T04:31:48.572 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4989.gce1b342b. 128/165 2024-09-17T04:31:48.590 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4989.gce1b342b.el9.n 130/165 2024-09-17T04:31:48.603 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4989.gce1b342b.el9.noa 129/165 2024-09-17T04:31:48.612 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4989.gce1 131/165 2024-09-17T04:31:48.725 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4989.gce1 131/165 2024-09-17T04:31:48.897 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4989.gce1b342b.el 132/165 2024-09-17T04:31:48.923 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-2:19.3.0-4989.gce1b342b.el9.x86_64 133/165 2024-09-17T04:31:48.946 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4989.gce1b342b.el9.x86_64 133/165 2024-09-17T04:31:48.947 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:31:48.947 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T04:31:48.947 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T04:31:48.947 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T04:31:48.947 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:31:48.990 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-rook-2:19.3.0-4989.gce1b342b.el9.noarch 134/165 2024-09-17T04:31:49.001 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4989.gce1b342b.el9.noarch 134/165 2024-09-17T04:31:49.100 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-2:19.3.0-4989.gce1b342b.el9.x86_64 135/165 2024-09-17T04:31:49.155 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 136/165 2024-09-17T04:31:50.115 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-17T04:31:50.142 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-radosgw-2:19.3.0-4989.gce1b342b.el9.x86_64 138/165 2024-09-17T04:31:50.165 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4989.gce1b342b.el9.x86_64 138/165 2024-09-17T04:31:50.166 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:31:50.166 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T04:31:50.166 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T04:31:50.166 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T04:31:50.166 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:31:50.210 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4989.gce1b3 139/165 2024-09-17T04:31:50.232 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4989.gce1b3 139/165 2024-09-17T04:31:50.232 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:31:50.233 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T04:31:50.233 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:31:50.285 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 140/165 2024-09-17T04:31:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 140/165 2024-09-17T04:31:50.308 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:31:50.308 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-17T04:31:50.308 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-17T04:31:50.309 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-17T04:31:50.309 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:31:50.628 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 141/165 2024-09-17T04:31:50.651 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 141/165 2024-09-17T04:31:50.651 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:31:50.651 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T04:31:50.651 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T04:31:50.651 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T04:31:50.651 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:31:51.991 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4989.gce1b342b.el9.noa 129/165 2024-09-17T04:31:52.059 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4989.gce1b342b.el9.n 130/165 2024-09-17T04:31:53.080 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4989.gce1b342b.el9.n 130/165 2024-09-17T04:31:53.109 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4989.gce1 131/165 2024-09-17T04:31:53.229 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4989.gce1 131/165 2024-09-17T04:31:53.401 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4989.gce1b342b.el 132/165 2024-09-17T04:31:53.436 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-2:19.3.0-4989.gce1b342b.el9.x86_64 133/165 2024-09-17T04:31:53.461 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4989.gce1b342b.el9.x86_64 133/165 2024-09-17T04:31:53.461 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:31:53.461 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T04:31:53.461 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T04:31:53.461 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T04:31:53.461 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:31:53.511 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-rook-2:19.3.0-4989.gce1b342b.el9.noarch 134/165 2024-09-17T04:31:53.524 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4989.gce1b342b.el9.noarch 134/165 2024-09-17T04:31:53.627 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-2:19.3.0-4989.gce1b342b.el9.x86_64 135/165 2024-09-17T04:31:53.680 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 136/165 2024-09-17T04:31:54.648 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-test-2:19.3.0-4989.gce1b342b.el9.x86_64 142/165 2024-09-17T04:31:54.698 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 143/165 2024-09-17T04:31:54.698 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-17T04:31:54.731 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-radosgw-2:19.3.0-4989.gce1b342b.el9.x86_64 138/165 2024-09-17T04:31:54.739 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-nbd-2:19.3.0-4989.gce1b342b.el9.x86_64 144/165 2024-09-17T04:31:54.756 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4989.gce1b342b.el9.x86_64 138/165 2024-09-17T04:31:54.756 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:31:54.756 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T04:31:54.756 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T04:31:54.756 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T04:31:54.756 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:31:54.770 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-17T04:31:54.798 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-17T04:31:54.798 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4989.gce1b3 139/165 2024-09-17T04:31:54.822 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4989.gce1b3 139/165 2024-09-17T04:31:54.822 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:31:54.822 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T04:31:54.823 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:31:54.873 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 140/165 2024-09-17T04:31:54.898 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 140/165 2024-09-17T04:31:54.898 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:31:54.898 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-17T04:31:54.898 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-17T04:31:54.899 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-17T04:31:54.899 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:31:54.985 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-shell-2:19.3.0-4989.gce1b342b.el9.x86_64 147/165 2024-09-17T04:31:55.036 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-17T04:31:55.075 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs-devel-2:19.3.0-4989.gce1b342b.el9.x86_ 149/165 2024-09-17T04:31:55.104 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-17T04:31:55.176 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-17T04:31:55.241 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 141/165 2024-09-17T04:31:55.265 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 141/165 2024-09-17T04:31:55.266 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T04:31:55.266 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T04:31:55.266 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T04:31:55.266 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T04:31:55.266 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:31:55.760 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-17T04:31:55.806 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-17T04:31:55.827 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-17T04:31:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-top-2:19.3.0-4989.gce1b342b.el9.noarch 153/165 2024-09-17T04:31:56.744 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-17T04:31:56.790 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-17T04:31:56.881 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-17T04:31:56.914 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-17T04:31:56.940 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-17T04:31:56.965 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-17T04:31:56.965 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-17T04:31:57.041 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-17T04:31:57.041 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-17T04:31:57.117 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-17T04:31:57.141 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-17T04:31:57.147 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-17T04:31:57.147 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-17T04:31:57.335 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-17T04:31:57.335 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-17T04:31:57.419 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-17T04:31:57.487 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-17T04:31:59.529 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-test-2:19.3.0-4989.gce1b342b.el9.x86_64 142/165 2024-09-17T04:31:59.570 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 143/165 2024-09-17T04:31:59.611 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-nbd-2:19.3.0-4989.gce1b342b.el9.x86_64 144/165 2024-09-17T04:31:59.642 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-17T04:31:59.670 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-17T04:31:59.870 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-shell-2:19.3.0-4989.gce1b342b.el9.x86_64 147/165 2024-09-17T04:31:59.916 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-17T04:31:59.938 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs-devel-2:19.3.0-4989.gce1b342b.el9.x86_ 149/165 2024-09-17T04:31:59.967 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-17T04:32:00.013 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-17T04:32:00.654 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-17T04:32:00.694 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-17T04:32:00.716 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-17T04:32:01.621 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-top-2:19.3.0-4989.gce1b342b.el9.noarch 153/165 2024-09-17T04:32:01.723 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-17T04:32:01.773 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-17T04:32:01.874 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-17T04:32:01.910 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-17T04:32:01.936 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-17T04:32:01.969 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-17T04:32:01.970 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-17T04:32:02.062 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-17T04:32:02.062 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-17T04:32:02.146 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-17T04:32:02.170 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-17T04:32:02.176 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-17T04:32:02.176 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-17T04:32:02.356 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-17T04:32:02.356 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-17T04:32:02.448 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-17T04:32:02.516 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-17T04:32:04.050 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-17T04:32:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-2:19.3.0-4989.gce1b342b.el9.x86_64 1/165 2024-09-17T04:32:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-base-2:19.3.0-4989.gce1b342b.el9.x86_64 2/165 2024-09-17T04:32:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 3/165 2024-09-17T04:32:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 4/165 2024-09-17T04:32:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4989.gce1b3 5/165 2024-09-17T04:32:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mds-2:19.3.0-4989.gce1b342b.el9.x86_64 6/165 2024-09-17T04:32:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-2:19.3.0-4989.gce1b342b.el9.x86_64 7/165 2024-09-17T04:32:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mon-2:19.3.0-4989.gce1b342b.el9.x86_64 8/165 2024-09-17T04:32:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-osd-2:19.3.0-4989.gce1b342b.el9.x86_64 9/165 2024-09-17T04:32:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-radosgw-2:19.3.0-4989.gce1b342b.el9.x86_64 10/165 2024-09-17T04:32:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-selinux-2:19.3.0-4989.gce1b342b.el9.x86_64 11/165 2024-09-17T04:32:04.051 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-test-2:19.3.0-4989.gce1b342b.el9.x86_64 12/165 2024-09-17T04:32:04.052 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 13/165 2024-09-17T04:32:04.052 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-shell-2:19.3.0-4989.gce1b342b.el9.x86_64 14/165 2024-09-17T04:32:04.052 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs-devel-2:19.3.0-4989.gce1b342b.el9.x86_ 15/165 2024-09-17T04:32:04.052 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs2-2:19.3.0-4989.gce1b342b.el9.x86_64 16/165 2024-09-17T04:32:04.052 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephsqlite-2:19.3.0-4989.gce1b342b.el9.x86_64 17/165 2024-09-17T04:32:04.052 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados-devel-2:19.3.0-4989.gce1b342b.el9.x86_6 18/165 2024-09-17T04:32:04.052 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libradosstriper1-2:19.3.0-4989.gce1b342b.el9.x86 19/165 2024-09-17T04:32:04.052 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librgw2-2:19.3.0-4989.gce1b342b.el9.x86_64 20/165 2024-09-17T04:32:04.052 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4989.gce1b342b.el 21/165 2024-09-17T04:32:04.052 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-common-2:19.3.0-4989.gce1b342b.el9. 22/165 2024-09-17T04:32:04.053 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cephfs-2:19.3.0-4989.gce1b342b.el9.x86_6 23/165 2024-09-17T04:32:04.053 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rados-2:19.3.0-4989.gce1b342b.el9.x86_64 24/165 2024-09-17T04:32:04.053 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rbd-2:19.3.0-4989.gce1b342b.el9.x86_64 25/165 2024-09-17T04:32:04.053 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rgw-2:19.3.0-4989.gce1b342b.el9.x86_64 26/165 2024-09-17T04:32:04.053 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 27/165 2024-09-17T04:32:04.053 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 28/165 2024-09-17T04:32:04.053 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-nbd-2:19.3.0-4989.gce1b342b.el9.x86_64 29/165 2024-09-17T04:32:04.053 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4989.gce1b342b. 30/165 2024-09-17T04:32:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4989.gce1b342b.el9.noa 31/165 2024-09-17T04:32:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4989.gce1b342b.el9.n 32/165 2024-09-17T04:32:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4989.gce1 33/165 2024-09-17T04:32:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4989.gce1b342b.el 34/165 2024-09-17T04:32:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4989.gce1b342b.el9.noarch 35/165 2024-09-17T04:32:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4989.gce1b342b.e 36/165 2024-09-17T04:32:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-volume-2:19.3.0-4989.gce1b342b.el9.noarch 37/165 2024-09-17T04:32:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 38/165 2024-09-17T04:32:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-top-2:19.3.0-4989.gce1b342b.el9.noarch 39/165 2024-09-17T04:32:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-17T04:32:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-17T04:32:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-17T04:32:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-17T04:32:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-17T04:32:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-17T04:32:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-17T04:32:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-17T04:32:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-17T04:32:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-17T04:32:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-17T04:32:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-17T04:32:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-17T04:32:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-17T04:32:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-17T04:32:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-17T04:32:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-17T04:32:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-17T04:32:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-17T04:32:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-17T04:32:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-17T04:32:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-17T04:32:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-17T04:32:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-17T04:32:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-17T04:32:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-17T04:32:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-17T04:32:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-17T04:32:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-17T04:32:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-17T04:32:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-17T04:32:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-17T04:32:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-17T04:32:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-17T04:32:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-17T04:32:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-17T04:32:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-17T04:32:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-17T04:32:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-17T04:32:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-17T04:32:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-17T04:32:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-17T04:32:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-17T04:32:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-17T04:32:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-17T04:32:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-17T04:32:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-17T04:32:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-17T04:32:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-17T04:32:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-17T04:32:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-17T04:32:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-17T04:32:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-17T04:32:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-17T04:32:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-17T04:32:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-17T04:32:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-17T04:32:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-17T04:32:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-17T04:32:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-17T04:32:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-17T04:32:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-17T04:32:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-17T04:32:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-17T04:32:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-17T04:32:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-17T04:32:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-17T04:32:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-17T04:32:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-17T04:32:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-17T04:32:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-17T04:32:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-17T04:32:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-17T04:32:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-17T04:32:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-17T04:32:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-17T04:32:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-17T04:32:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-17T04:32:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-17T04:32:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-17T04:32:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-17T04:32:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-17T04:32:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-17T04:32:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-17T04:32:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-17T04:32:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-17T04:32:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-17T04:32:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-17T04:32:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-17T04:32:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-17T04:32:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-17T04:32:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-17T04:32:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-17T04:32:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-17T04:32:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-17T04:32:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-17T04:32:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-17T04:32:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-17T04:32:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-17T04:32:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-17T04:32:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-17T04:32:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-17T04:32:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-17T04:32:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-17T04:32:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-17T04:32:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-17T04:32:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-17T04:32:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-17T04:32:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-17T04:32:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-17T04:32:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:19.3.0-4989.gce1b342b.el9.x86_64 150/165 2024-09-17T04:32:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-17T04:32:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:19.3.0-4989.gce1b342b.el9.x86_64 152/165 2024-09-17T04:32:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-17T04:32:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-17T04:32:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-17T04:32:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-17T04:32:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-17T04:32:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-17T04:32:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-17T04:32:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-17T04:32:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-17T04:32:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-17T04:32:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-17T04:32:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-17T04:32:06.692 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-17T04:32:06.692 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:06.693 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2024-09-17T04:32:06.693 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-17T04:32:06.693 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-17T04:32:06.693 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-17T04:32:06.693 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-17T04:32:06.693 INFO:teuthology.orchestra.run.smithi092.stdout: librados2-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.693 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.693 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-17T04:32:06.693 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-17T04:32:06.693 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-17T04:32:06.693 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-17T04:32:06.693 INFO:teuthology.orchestra.run.smithi092.stdout: automake-1.16.2-8.el9.noarch 2024-09-17T04:32:06.694 INFO:teuthology.orchestra.run.smithi092.stdout: bison-3.7.4-5.el9.x86_64 2024-09-17T04:32:06.694 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T04:32:06.694 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.694 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.694 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.694 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.694 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:32:06.694 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.694 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.694 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.694 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:32:06.694 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:32:06.695 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:32:06.695 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:32:06.695 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:32:06.695 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.695 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.695 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:32:06.695 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.695 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.695 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.695 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:32:06.695 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:32:06.695 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.695 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-shell-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.695 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:32:06.696 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-17T04:32:06.696 INFO:teuthology.orchestra.run.smithi092.stdout: flex-2.6.4-9.el9.x86_64 2024-09-17T04:32:06.696 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T04:32:06.696 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T04:32:06.696 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T04:32:06.696 INFO:teuthology.orchestra.run.smithi092.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-17T04:32:06.696 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T04:32:06.696 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-17T04:32:06.696 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-17T04:32:06.696 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T04:32:06.697 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T04:32:06.697 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.697 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.697 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.697 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T04:32:06.697 INFO:teuthology.orchestra.run.smithi092.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-17T04:32:06.697 INFO:teuthology.orchestra.run.smithi092.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T04:32:06.697 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T04:32:06.697 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T04:32:06.697 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T04:32:06.697 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.697 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.697 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T04:32:06.697 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.698 INFO:teuthology.orchestra.run.smithi092.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-17T04:32:06.698 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T04:32:06.698 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-17T04:32:06.698 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-17T04:32:06.698 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T04:32:06.698 INFO:teuthology.orchestra.run.smithi092.stdout: lua-5.4.4-4.el9.x86_64 2024-09-17T04:32:06.698 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-17T04:32:06.698 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-17T04:32:06.698 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T04:32:06.698 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T04:32:06.698 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-17T04:32:06.698 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T04:32:06.698 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-13.16-1.el9.x86_64 2024-09-17T04:32:06.699 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-17T04:32:06.699 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-17T04:32:06.699 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-17T04:32:06.699 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-17T04:32:06.699 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-17T04:32:06.699 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T04:32:06.699 INFO:teuthology.orchestra.run.smithi092.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-17T04:32:06.699 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T04:32:06.699 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T04:32:06.699 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T04:32:06.699 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.699 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.699 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.700 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T04:32:06.700 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T04:32:06.700 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T04:32:06.700 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T04:32:06.700 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-17T04:32:06.700 INFO:teuthology.orchestra.run.smithi092.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-17T04:32:06.700 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T04:32:06.700 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T04:32:06.700 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T04:32:06.700 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-17T04:32:06.701 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-17T04:32:06.701 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T04:32:06.701 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T04:32:06.701 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T04:32:06.701 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T04:32:06.701 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T04:32:06.701 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T04:32:06.701 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T04:32:06.701 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T04:32:06.702 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T04:32:06.702 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T04:32:06.702 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T04:32:06.702 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T04:32:06.702 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T04:32:06.702 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T04:32:06.702 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T04:32:06.702 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T04:32:06.702 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T04:32:06.702 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T04:32:06.702 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T04:32:06.702 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T04:32:06.702 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T04:32:06.702 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T04:32:06.703 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-17T04:32:06.703 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T04:32:06.703 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T04:32:06.703 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T04:32:06.703 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T04:32:06.703 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-17T04:32:06.703 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T04:32:06.703 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.703 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.703 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T04:32:06.704 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T04:32:06.704 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T04:32:06.704 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.704 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T04:32:06.704 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T04:32:06.704 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T04:32:06.704 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T04:32:06.704 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T04:32:06.705 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T04:32:06.705 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T04:32:06.705 INFO:teuthology.orchestra.run.smithi092.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-17T04:32:06.705 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T04:32:06.705 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T04:32:06.705 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T04:32:06.705 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T04:32:06.705 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-17T04:32:06.705 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-17T04:32:06.705 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-17T04:32:06.705 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.705 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.706 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:06.706 INFO:teuthology.orchestra.run.smithi092.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T04:32:06.706 INFO:teuthology.orchestra.run.smithi092.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T04:32:06.706 INFO:teuthology.orchestra.run.smithi092.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T04:32:06.706 INFO:teuthology.orchestra.run.smithi092.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-17T04:32:06.706 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-17T04:32:06.706 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T04:32:06.706 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-17T04:32:06.706 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:06.707 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-17T04:32:07.108 DEBUG:teuthology.parallel:result is None 2024-09-17T04:32:08.603 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-17T04:32:08.603 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-2:19.3.0-4989.gce1b342b.el9.x86_64 1/165 2024-09-17T04:32:08.603 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-base-2:19.3.0-4989.gce1b342b.el9.x86_64 2/165 2024-09-17T04:32:08.603 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 3/165 2024-09-17T04:32:08.603 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 4/165 2024-09-17T04:32:08.603 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4989.gce1b3 5/165 2024-09-17T04:32:08.603 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mds-2:19.3.0-4989.gce1b342b.el9.x86_64 6/165 2024-09-17T04:32:08.603 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-2:19.3.0-4989.gce1b342b.el9.x86_64 7/165 2024-09-17T04:32:08.603 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mon-2:19.3.0-4989.gce1b342b.el9.x86_64 8/165 2024-09-17T04:32:08.604 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-osd-2:19.3.0-4989.gce1b342b.el9.x86_64 9/165 2024-09-17T04:32:08.604 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-radosgw-2:19.3.0-4989.gce1b342b.el9.x86_64 10/165 2024-09-17T04:32:08.604 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-selinux-2:19.3.0-4989.gce1b342b.el9.x86_64 11/165 2024-09-17T04:32:08.604 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-test-2:19.3.0-4989.gce1b342b.el9.x86_64 12/165 2024-09-17T04:32:08.604 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 13/165 2024-09-17T04:32:08.604 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-shell-2:19.3.0-4989.gce1b342b.el9.x86_64 14/165 2024-09-17T04:32:08.604 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs-devel-2:19.3.0-4989.gce1b342b.el9.x86_ 15/165 2024-09-17T04:32:08.604 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs2-2:19.3.0-4989.gce1b342b.el9.x86_64 16/165 2024-09-17T04:32:08.604 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephsqlite-2:19.3.0-4989.gce1b342b.el9.x86_64 17/165 2024-09-17T04:32:08.604 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados-devel-2:19.3.0-4989.gce1b342b.el9.x86_6 18/165 2024-09-17T04:32:08.604 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libradosstriper1-2:19.3.0-4989.gce1b342b.el9.x86 19/165 2024-09-17T04:32:08.604 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librgw2-2:19.3.0-4989.gce1b342b.el9.x86_64 20/165 2024-09-17T04:32:08.604 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4989.gce1b342b.el 21/165 2024-09-17T04:32:08.605 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-common-2:19.3.0-4989.gce1b342b.el9. 22/165 2024-09-17T04:32:08.605 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cephfs-2:19.3.0-4989.gce1b342b.el9.x86_6 23/165 2024-09-17T04:32:08.605 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rados-2:19.3.0-4989.gce1b342b.el9.x86_64 24/165 2024-09-17T04:32:08.605 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rbd-2:19.3.0-4989.gce1b342b.el9.x86_64 25/165 2024-09-17T04:32:08.605 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rgw-2:19.3.0-4989.gce1b342b.el9.x86_64 26/165 2024-09-17T04:32:08.605 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 27/165 2024-09-17T04:32:08.605 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 28/165 2024-09-17T04:32:08.605 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-nbd-2:19.3.0-4989.gce1b342b.el9.x86_64 29/165 2024-09-17T04:32:08.621 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4989.gce1b342b. 30/165 2024-09-17T04:32:08.621 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4989.gce1b342b.el9.noa 31/165 2024-09-17T04:32:08.621 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4989.gce1b342b.el9.n 32/165 2024-09-17T04:32:08.621 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4989.gce1 33/165 2024-09-17T04:32:08.621 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4989.gce1b342b.el 34/165 2024-09-17T04:32:08.622 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4989.gce1b342b.el9.noarch 35/165 2024-09-17T04:32:08.622 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4989.gce1b342b.e 36/165 2024-09-17T04:32:08.622 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-volume-2:19.3.0-4989.gce1b342b.el9.noarch 37/165 2024-09-17T04:32:08.622 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 38/165 2024-09-17T04:32:08.622 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-top-2:19.3.0-4989.gce1b342b.el9.noarch 39/165 2024-09-17T04:32:08.622 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-17T04:32:08.622 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-17T04:32:08.622 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-17T04:32:08.622 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-17T04:32:08.622 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-17T04:32:08.622 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-17T04:32:08.622 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-17T04:32:08.622 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-17T04:32:08.623 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-17T04:32:08.623 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-17T04:32:08.623 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-17T04:32:08.623 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-17T04:32:08.623 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-17T04:32:08.623 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-17T04:32:08.623 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-17T04:32:08.623 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-17T04:32:08.623 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-17T04:32:08.623 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-17T04:32:08.623 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-17T04:32:08.623 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-17T04:32:08.623 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-17T04:32:08.624 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-17T04:32:08.624 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-17T04:32:08.624 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-17T04:32:08.624 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-17T04:32:08.624 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-17T04:32:08.624 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-17T04:32:08.624 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-17T04:32:08.624 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-17T04:32:08.624 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-17T04:32:08.624 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-17T04:32:08.624 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-17T04:32:08.624 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-17T04:32:08.624 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-17T04:32:08.625 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-17T04:32:08.625 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-17T04:32:08.625 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-17T04:32:08.625 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-17T04:32:08.625 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-17T04:32:08.625 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-17T04:32:08.625 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-17T04:32:08.625 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-17T04:32:08.625 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-17T04:32:08.625 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-17T04:32:08.625 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-17T04:32:08.625 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-17T04:32:08.625 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-17T04:32:08.625 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-17T04:32:08.626 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-17T04:32:08.626 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-17T04:32:08.626 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-17T04:32:08.626 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-17T04:32:08.626 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-17T04:32:08.626 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-17T04:32:08.626 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-17T04:32:08.626 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-17T04:32:08.626 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-17T04:32:08.626 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-17T04:32:08.626 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-17T04:32:08.626 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-17T04:32:08.626 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-17T04:32:08.627 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-17T04:32:08.627 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-17T04:32:08.627 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-17T04:32:08.627 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-17T04:32:08.627 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-17T04:32:08.627 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-17T04:32:08.627 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-17T04:32:08.627 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-17T04:32:08.627 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-17T04:32:08.627 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-17T04:32:08.627 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-17T04:32:08.627 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-17T04:32:08.627 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-17T04:32:08.628 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-17T04:32:08.628 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-17T04:32:08.628 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-17T04:32:08.628 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-17T04:32:08.628 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-17T04:32:08.628 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-17T04:32:08.628 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-17T04:32:08.628 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-17T04:32:08.628 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-17T04:32:08.628 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-17T04:32:08.628 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-17T04:32:08.628 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-17T04:32:08.628 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-17T04:32:08.628 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-17T04:32:08.629 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-17T04:32:08.629 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-17T04:32:08.629 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-17T04:32:08.629 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-17T04:32:08.629 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-17T04:32:08.629 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-17T04:32:08.629 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-17T04:32:08.629 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-17T04:32:08.629 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-17T04:32:08.629 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-17T04:32:08.629 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-17T04:32:08.629 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-17T04:32:08.629 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-17T04:32:08.630 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-17T04:32:08.630 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-17T04:32:08.630 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-17T04:32:08.630 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-17T04:32:08.630 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-17T04:32:08.630 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-17T04:32:08.630 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-17T04:32:08.630 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-17T04:32:08.630 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-17T04:32:08.630 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:19.3.0-4989.gce1b342b.el9.x86_64 150/165 2024-09-17T04:32:08.630 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-17T04:32:08.631 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:19.3.0-4989.gce1b342b.el9.x86_64 152/165 2024-09-17T04:32:08.631 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-17T04:32:08.631 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-17T04:32:08.631 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-17T04:32:08.631 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-17T04:32:08.631 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-17T04:32:08.631 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-17T04:32:08.631 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-17T04:32:08.631 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-17T04:32:08.631 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-17T04:32:08.631 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-17T04:32:08.631 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-17T04:32:08.632 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-17T04:32:11.270 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-17T04:32:11.270 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:11.270 INFO:teuthology.orchestra.run.smithi037.stdout:Upgraded: 2024-09-17T04:32:11.270 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-17T04:32:11.270 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-17T04:32:11.270 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-17T04:32:11.270 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-17T04:32:11.270 INFO:teuthology.orchestra.run.smithi037.stdout: librados2-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.270 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.271 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-17T04:32:11.271 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-17T04:32:11.271 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-09-17T04:32:11.271 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-17T04:32:11.271 INFO:teuthology.orchestra.run.smithi037.stdout: automake-1.16.2-8.el9.noarch 2024-09-17T04:32:11.271 INFO:teuthology.orchestra.run.smithi037.stdout: bison-3.7.4-5.el9.x86_64 2024-09-17T04:32:11.271 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T04:32:11.271 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.271 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.271 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.271 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.271 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:32:11.272 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.272 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.272 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.272 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:32:11.272 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:32:11.272 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:32:11.272 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:32:11.272 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:32:11.272 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.272 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.272 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:32:11.272 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.272 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.273 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.273 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:32:11.273 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:32:11.273 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.273 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.273 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T04:32:11.273 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-17T04:32:11.273 INFO:teuthology.orchestra.run.smithi037.stdout: flex-2.6.4-9.el9.x86_64 2024-09-17T04:32:11.273 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T04:32:11.273 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T04:32:11.273 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T04:32:11.274 INFO:teuthology.orchestra.run.smithi037.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-17T04:32:11.274 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T04:32:11.274 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-17T04:32:11.274 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-17T04:32:11.274 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T04:32:11.274 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T04:32:11.274 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.274 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.274 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.275 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T04:32:11.275 INFO:teuthology.orchestra.run.smithi037.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-17T04:32:11.275 INFO:teuthology.orchestra.run.smithi037.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T04:32:11.275 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T04:32:11.275 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T04:32:11.275 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T04:32:11.275 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.275 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.275 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T04:32:11.275 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.275 INFO:teuthology.orchestra.run.smithi037.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-17T04:32:11.276 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T04:32:11.276 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-17T04:32:11.276 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-17T04:32:11.276 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T04:32:11.276 INFO:teuthology.orchestra.run.smithi037.stdout: lua-5.4.4-4.el9.x86_64 2024-09-17T04:32:11.276 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-17T04:32:11.276 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-17T04:32:11.276 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T04:32:11.276 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T04:32:11.276 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-17T04:32:11.276 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T04:32:11.276 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-13.16-1.el9.x86_64 2024-09-17T04:32:11.277 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-17T04:32:11.277 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-17T04:32:11.277 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-17T04:32:11.277 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-17T04:32:11.277 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-17T04:32:11.277 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T04:32:11.277 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-17T04:32:11.277 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T04:32:11.277 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T04:32:11.277 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T04:32:11.277 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.277 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.277 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.278 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T04:32:11.278 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T04:32:11.278 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T04:32:11.278 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T04:32:11.278 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-17T04:32:11.278 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-17T04:32:11.278 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T04:32:11.278 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T04:32:11.278 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T04:32:11.278 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-17T04:32:11.279 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-17T04:32:11.279 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T04:32:11.279 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T04:32:11.279 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T04:32:11.279 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T04:32:11.279 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T04:32:11.279 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T04:32:11.279 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T04:32:11.279 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T04:32:11.279 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T04:32:11.279 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T04:32:11.279 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T04:32:11.280 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T04:32:11.280 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T04:32:11.280 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T04:32:11.280 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T04:32:11.280 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T04:32:11.280 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T04:32:11.280 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T04:32:11.280 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T04:32:11.280 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T04:32:11.280 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T04:32:11.280 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T04:32:11.280 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-17T04:32:11.280 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T04:32:11.281 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T04:32:11.281 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T04:32:11.281 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T04:32:11.281 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-17T04:32:11.281 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T04:32:11.281 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.281 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.281 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T04:32:11.281 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T04:32:11.281 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T04:32:11.281 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.281 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T04:32:11.282 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T04:32:11.282 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T04:32:11.282 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T04:32:11.282 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T04:32:11.282 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T04:32:11.282 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T04:32:11.282 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-17T04:32:11.282 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T04:32:11.282 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T04:32:11.282 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T04:32:11.282 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T04:32:11.282 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-17T04:32:11.282 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-17T04:32:11.283 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-17T04:32:11.283 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.283 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.283 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T04:32:11.283 INFO:teuthology.orchestra.run.smithi037.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T04:32:11.283 INFO:teuthology.orchestra.run.smithi037.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T04:32:11.283 INFO:teuthology.orchestra.run.smithi037.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T04:32:11.283 INFO:teuthology.orchestra.run.smithi037.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-17T04:32:11.283 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-17T04:32:11.283 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T04:32:11.284 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-17T04:32:11.284 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:11.284 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T04:32:11.686 DEBUG:teuthology.parallel:result is None 2024-09-17T04:32:11.687 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T04:32:11.833 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T04:32:11.856 INFO:teuthology.orchestra.run.smithi037.stdout:19.3.0-4989.gce1b342b.el9 2024-09-17T04:32:12.490 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4989.gce1b342b.el9 2024-09-17T04:32:12.491 INFO:teuthology.task.install:The correct ceph version 19.3.0-4989.gce1b342b is installed. 2024-09-17T04:32:12.493 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T04:32:12.638 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T04:32:12.664 INFO:teuthology.orchestra.run.smithi092.stdout:19.3.0-4989.gce1b342b.el9 2024-09-17T04:32:12.664 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4989.gce1b342b.el9 2024-09-17T04:32:12.664 INFO:teuthology.task.install:The correct ceph version 19.3.0-4989.gce1b342b is installed. 2024-09-17T04:32:12.666 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-17T04:32:12.666 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:32:12.667 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T04:32:12.700 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:32:12.701 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T04:32:12.735 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-17T04:32:12.735 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:32:12.736 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T04:32:12.772 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T04:32:12.842 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:32:12.843 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T04:32:12.870 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T04:32:12.937 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-17T04:32:12.937 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:32:12.937 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T04:32:12.966 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T04:32:13.034 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:32:13.034 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T04:32:13.061 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T04:32:13.126 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-17T04:32:13.127 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:32:13.127 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T04:32:13.156 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T04:32:13.224 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:32:13.224 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T04:32:13.250 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T04:32:13.316 INFO:teuthology.run_tasks:Running task ceph... 2024-09-17T04:32:13.648 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-17T04:32:13.649 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 777 /var/log/ceph 2024-09-17T04:32:13.651 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /var/log/ceph 2024-09-17T04:32:13.678 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-17T04:32:13.678 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T04:32:13.721 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T04:32:13.748 INFO:tasks.ceph:Creating extra log directories... 2024-09-17T04:32:13.748 DEBUG:teuthology.orchestra.run.smithi037:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T04:32:13.790 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T04:32:13.827 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-17T04:32:13.827 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-17T04:32:13.828 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-17T04:32:13.828 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:32:13.828 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-17T04:32:13.874 DEBUG:teuthology.orchestra.run.smithi037:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-17T04:32:13.990 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-17T04:32:14.017 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:32:14.018 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-17T04:32:14.036 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-17T04:32:14.150 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-17T04:32:14.181 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-17T04:32:14.181 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'MDS_FAILED', 'MDS_DAMAGE'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-17T04:32:14.181 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908540', 'branch': 'wip-pdonnell-testing-20240916.200549-debug', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/alternate-pool}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7908540', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240916.200549-debug'}, 'ceph': {'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'MDS_FAILED', 'MDS_DAMAGE'], 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240916.200549-debug', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 90, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 476, 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf', 'sleep_before_teardown': 0, 'subset': '320/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240916.200549-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi037.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNy/At/D3by+/dEkkI/d45LiBQBqS6hLJSrxvosT7nQ34twYHz8GmAoOr+oKhAtE6gMJ0VL9j8Ozjqt5LlYMxgs=', 'smithi092.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEc66UZbUU/s6SQrv/Lc7fz5B2DLhLKpQNexSMDp04LwzgcAKVrZ2W9OsDBPTemVtL2NWDEZvwiBwSYlgoV/69E='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}}, {'ceph': {'create_rbd_pool': False, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'MDS_FAILED', 'MDS_DAMAGE'], 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_recovery_pool']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-17_02:09:51', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-17T04:32:14.182 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T04:32:14.184 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T04:32:14.233 DEBUG:teuthology.orchestra.run.smithi037:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T04:32:14.241 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T04:32:14.299 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:32:14.300 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T04:32:14.321 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T04:32:14.321 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_1 2024-09-17T04:32:14.377 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T04:32:14.378 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T04:32:14.378 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-17T04:32:14.378 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T04:32:14.378 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T04:32:14.378 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-09-17 04:32:02.875780092 +0000 2024-09-17T04:32:14.378 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-09-17 04:30:11.186736087 +0000 2024-09-17T04:32:14.378 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-09-17 04:30:11.186736087 +0000 2024-09-17T04:32:14.378 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-09-17 04:30:11.186736087 +0000 2024-09-17T04:32:14.378 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T04:32:14.441 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-09-17T04:32:14.442 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-09-17T04:32:14.442 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000185766 s, 2.8 MB/s 2024-09-17T04:32:14.443 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T04:32:14.499 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_2 2024-09-17T04:32:14.557 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T04:32:14.558 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T04:32:14.558 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-17T04:32:14.558 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T04:32:14.558 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T04:32:14.558 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-09-17 04:32:02.875780092 +0000 2024-09-17T04:32:14.558 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-09-17 04:30:11.510731438 +0000 2024-09-17T04:32:14.558 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-09-17 04:30:11.510731438 +0000 2024-09-17T04:32:14.558 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-09-17 04:30:11.510731438 +0000 2024-09-17T04:32:14.559 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T04:32:14.621 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-09-17T04:32:14.621 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-09-17T04:32:14.621 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000195115 s, 2.6 MB/s 2024-09-17T04:32:14.623 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T04:32:14.679 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_3 2024-09-17T04:32:14.735 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T04:32:14.735 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T04:32:14.735 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-17T04:32:14.735 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T04:32:14.735 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T04:32:14.735 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-09-17 04:32:02.876780077 +0000 2024-09-17T04:32:14.735 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-09-17 04:30:11.821726975 +0000 2024-09-17T04:32:14.735 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-09-17 04:30:11.821726975 +0000 2024-09-17T04:32:14.735 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-09-17 04:30:11.821726975 +0000 2024-09-17T04:32:14.736 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T04:32:14.800 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-09-17T04:32:14.800 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-09-17T04:32:14.800 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000213282 s, 2.4 MB/s 2024-09-17T04:32:14.801 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T04:32:14.858 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_4 2024-09-17T04:32:14.914 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T04:32:14.914 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T04:32:14.914 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-17T04:32:14.914 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T04:32:14.914 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T04:32:14.915 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-09-17 04:32:02.876780077 +0000 2024-09-17T04:32:14.915 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-09-17 04:30:12.144722340 +0000 2024-09-17T04:32:14.915 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-09-17 04:30:12.144722340 +0000 2024-09-17T04:32:14.915 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-09-17 04:30:12.144722340 +0000 2024-09-17T04:32:14.915 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T04:32:14.978 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-09-17T04:32:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-09-17T04:32:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.00018133 s, 2.8 MB/s 2024-09-17T04:32:14.980 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T04:32:15.037 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T04:32:15.037 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:32:15.037 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T04:32:15.056 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T04:32:15.056 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2024-09-17T04:32:15.115 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T04:32:15.115 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T04:32:15.116 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-17T04:32:15.116 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T04:32:15.116 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T04:32:15.116 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-17 04:31:57.854058002 +0000 2024-09-17T04:32:15.116 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-17 04:30:06.469438747 +0000 2024-09-17T04:32:15.116 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-17 04:30:06.469438747 +0000 2024-09-17T04:32:15.116 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-17 04:30:06.469438747 +0000 2024-09-17T04:32:15.116 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T04:32:15.183 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-17T04:32:15.183 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-17T04:32:15.183 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000223416 s, 2.3 MB/s 2024-09-17T04:32:15.185 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T04:32:15.245 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2024-09-17T04:32:15.304 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T04:32:15.304 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T04:32:15.304 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-17T04:32:15.304 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T04:32:15.304 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T04:32:15.304 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-17 04:31:57.855057973 +0000 2024-09-17T04:32:15.304 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-17 04:30:06.786429431 +0000 2024-09-17T04:32:15.304 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-17 04:30:06.786429431 +0000 2024-09-17T04:32:15.304 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-17 04:30:06.786429431 +0000 2024-09-17T04:32:15.305 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T04:32:15.372 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-17T04:32:15.372 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-17T04:32:15.372 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000263628 s, 1.9 MB/s 2024-09-17T04:32:15.377 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T04:32:15.432 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2024-09-17T04:32:15.490 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T04:32:15.490 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T04:32:15.490 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-17T04:32:15.490 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T04:32:15.491 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T04:32:15.491 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-17 04:31:57.855057973 +0000 2024-09-17T04:32:15.491 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-17 04:30:07.113419822 +0000 2024-09-17T04:32:15.491 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-17 04:30:07.113419822 +0000 2024-09-17T04:32:15.491 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-17 04:30:07.113419822 +0000 2024-09-17T04:32:15.491 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T04:32:15.557 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-17T04:32:15.557 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-17T04:32:15.557 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000178382 s, 2.9 MB/s 2024-09-17T04:32:15.558 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T04:32:15.617 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2024-09-17T04:32:15.675 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T04:32:15.675 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T04:32:15.675 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-17T04:32:15.675 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T04:32:15.675 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T04:32:15.675 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-17 04:31:57.855057973 +0000 2024-09-17T04:32:15.675 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-17 04:30:07.433410420 +0000 2024-09-17T04:32:15.676 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-17 04:30:07.433410420 +0000 2024-09-17T04:32:15.676 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-17 04:30:07.433410420 +0000 2024-09-17T04:32:15.676 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T04:32:15.741 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-17T04:32:15.741 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-17T04:32:15.741 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.0002016 s, 2.5 MB/s 2024-09-17T04:32:15.743 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T04:32:15.800 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T04:32:15.801 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi037.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-17T04:32:15.801 INFO:tasks.ceph:Generating config... 2024-09-17T04:32:15.802 INFO:tasks.ceph:[client] pid file = 2024-09-17T04:32:15.802 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-17T04:32:15.802 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-17T04:32:15.802 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-17T04:32:15.802 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-17T04:32:15.802 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-17T04:32:15.802 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-17T04:32:15.802 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-17T04:32:15.802 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-17T04:32:15.803 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-17T04:32:15.803 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-17T04:32:15.803 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-17T04:32:15.803 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-17T04:32:15.803 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-17T04:32:15.803 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-17T04:32:15.803 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-17T04:32:15.803 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-17T04:32:15.803 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-17T04:32:15.803 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-17T04:32:15.803 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-17T04:32:15.803 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-17T04:32:15.804 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-17T04:32:15.804 INFO:tasks.ceph:Setting up mon.a... 2024-09-17T04:32:15.804 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-17T04:32:15.867 INFO:teuthology.orchestra.run.smithi037.stdout:creating /etc/ceph/ceph.keyring 2024-09-17T04:32:15.870 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-17T04:32:15.927 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T04:32:15.953 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.37'), ('mon.b', '172.21.15.92'), ('mon.c', '[v2:172.21.15.92:3301,v1:172.21.15.92:6790]')] 2024-09-17T04:32:15.954 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.37,172.21.15.92,[v2:172.21.15.92:3301,v1:172.21.15.92:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-17T04:32:15.954 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:32:15.955 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T04:32:16.010 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.37 --add b 172.21.15.92 --addv c '[v2:172.21.15.92:3301,v1:172.21.15.92:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:32:16.108 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:32:16.108 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool: generated fsid a908a603-430c-4209-8294-0c3460b35f42 2024-09-17T04:32:16.108 INFO:teuthology.orchestra.run.smithi037.stdout:setting min_mon_release = quincy 2024-09-17T04:32:16.108 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 0 2024-09-17T04:32:16.108 INFO:teuthology.orchestra.run.smithi037.stdout:fsid a908a603-430c-4209-8294-0c3460b35f42 2024-09-17T04:32:16.108 INFO:teuthology.orchestra.run.smithi037.stdout:last_changed 2024-09-17T04:32:16.109139+0000 2024-09-17T04:32:16.108 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-09-17T04:32:16.109139+0000 2024-09-17T04:32:16.108 INFO:teuthology.orchestra.run.smithi037.stdout:min_mon_release 17 (quincy) 2024-09-17T04:32:16.108 INFO:teuthology.orchestra.run.smithi037.stdout:election_strategy: 1 2024-09-17T04:32:16.108 INFO:teuthology.orchestra.run.smithi037.stdout:0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-09-17T04:32:16.108 INFO:teuthology.orchestra.run.smithi037.stdout:1: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.b 2024-09-17T04:32:16.109 INFO:teuthology.orchestra.run.smithi037.stdout:2: [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] mon.c 2024-09-17T04:32:16.109 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-17T04:32:16.110 DEBUG:teuthology.orchestra.run.smithi037:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T04:32:16.125 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID a908a603-430c-4209-8294-0c3460b35f42... 2024-09-17T04:32:16.126 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T04:32:16.168 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T04:32:16.211 INFO:teuthology.orchestra.run.smithi037.stdout:[global] 2024-09-17T04:32:16.211 INFO:teuthology.orchestra.run.smithi037.stdout: chdir = "" 2024-09-17T04:32:16.211 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T04:32:16.211 INFO:teuthology.orchestra.run.smithi037.stdout: auth supported = cephx 2024-09-17T04:32:16.211 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.211 INFO:teuthology.orchestra.run.smithi037.stdout: filestore xattr use omap = true 2024-09-17T04:32:16.211 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.212 INFO:teuthology.orchestra.run.smithi037.stdout: mon clock drift allowed = 1.000 2024-09-17T04:32:16.212 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.212 INFO:teuthology.orchestra.run.smithi037.stdout: osd crush chooseleaf type = 0 2024-09-17T04:32:16.212 INFO:teuthology.orchestra.run.smithi037.stdout: auth debug = true 2024-09-17T04:32:16.212 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.212 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on old message = true 2024-09-17T04:32:16.213 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on bug = true 2024-09-17T04:32:16.213 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.213 INFO:teuthology.orchestra.run.smithi037.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T04:32:16.213 INFO:teuthology.orchestra.run.smithi037.stdout: mon pg warn max object skew = 0 2024-09-17T04:32:16.213 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.213 INFO:teuthology.orchestra.run.smithi037.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T04:32:16.213 INFO:teuthology.orchestra.run.smithi037.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T04:32:16.213 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default size = 2 2024-09-17T04:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow primary affinity = true 2024-09-17T04:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow pg remap = true 2024-09-17T04:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on legacy crush tunables = false 2024-09-17T04:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on crush straw calc version zero = false 2024-09-17T04:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on no sortbitwise = false 2024-09-17T04:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on osd down out interval zero = false 2024-09-17T04:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on too few osds = false 2024-09-17T04:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T04:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T04:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: mon_allow_pool_size_one = true 2024-09-17T04:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T04:32:16.215 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.215 INFO:teuthology.orchestra.run.smithi037.stdout: osd default data pool replay window = 5 2024-09-17T04:32:16.215 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.215 INFO:teuthology.orchestra.run.smithi037.stdout: mon allow pool delete = true 2024-09-17T04:32:16.215 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.215 INFO:teuthology.orchestra.run.smithi037.stdout: mon cluster log file level = debug 2024-09-17T04:32:16.215 INFO:teuthology.orchestra.run.smithi037.stdout: debug asserts on shutdown = true 2024-09-17T04:32:16.215 INFO:teuthology.orchestra.run.smithi037.stdout: mon health detail to clog = false 2024-09-17T04:32:16.215 INFO:teuthology.orchestra.run.smithi037.stdout: mon host = "172.21.15.37,172.21.15.92,[v2:172.21.15.92:3301,v1:172.21.15.92:6790]" 2024-09-17T04:32:16.215 INFO:teuthology.orchestra.run.smithi037.stdout: fsid = a908a603-430c-4209-8294-0c3460b35f42 2024-09-17T04:32:16.215 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.215 INFO:teuthology.orchestra.run.smithi037.stdout:[osd] 2024-09-17T04:32:16.215 INFO:teuthology.orchestra.run.smithi037.stdout: osd journal size = 100 2024-09-17T04:32:16.215 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub load threshold = 5.0 2024-09-17T04:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub max interval = 600 2024-09-17T04:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: osd mclock profile = high_recovery_ops 2024-09-17T04:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: osd recover clone overlap = true 2024-09-17T04:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: osd recovery max chunk = 1048576 2024-09-17T04:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug shutdown = true 2024-09-17T04:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug op order = true 2024-09-17T04:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug verify stray on activate = true 2024-09-17T04:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug trim objects = true 2024-09-17T04:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: osd open classes on start = true 2024-09-17T04:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug pg log writeout = true 2024-09-17T04:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: osd deep scrub update digest min age = 30 2024-09-17T04:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: osd map max advance = 10 2024-09-17T04:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: journal zero on create = true 2024-09-17T04:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: filestore ondisk finisher threads = 3 2024-09-17T04:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: filestore apply finisher threads = 3 2024-09-17T04:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: bdev debug aio = true 2024-09-17T04:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug misdirected ops = true 2024-09-17T04:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: bdev async discard = True 2024-09-17T04:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: bdev enable discard = True 2024-09-17T04:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore allocator = bitmap 2024-09-17T04:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore block size = 96636764160 2024-09-17T04:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore fsck on mount = True 2024-09-17T04:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluefs = 1/20 2024-09-17T04:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluestore = 1/20 2024-09-17T04:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-09-17T04:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: debug osd = 20 2024-09-17T04:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: debug rocksdb = 4/10 2024-09-17T04:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T04:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd full ratio = 0.9 2024-09-17T04:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd nearfull ratio = 0.8 2024-09-17T04:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: osd failsafe full ratio = 0.95 2024-09-17T04:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: osd objectstore = bluestore 2024-09-17T04:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr] 2024-09-17T04:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-09-17T04:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: debug mgr = 20 2024-09-17T04:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-09-17T04:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-09-17T04:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-09-17T04:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-09-17T04:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: mgr/telemetry/nag = false 2024-09-17T04:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout:[mon] 2024-09-17T04:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-09-17T04:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-09-17T04:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: debug paxos = 20 2024-09-17T04:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-09-17T04:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: mon data avail warn = 5 2024-09-17T04:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: mon mgr mkfs grace = 240 2024-09-17T04:32:16.220 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-09-17T04:32:16.220 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd reporter subtree level = osd 2024-09-17T04:32:16.220 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd prime pg temp = true 2024-09-17T04:32:16.220 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-09-17T04:32:16.220 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.220 INFO:teuthology.orchestra.run.smithi037.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T04:32:16.220 INFO:teuthology.orchestra.run.smithi037.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T04:32:16.220 INFO:teuthology.orchestra.run.smithi037.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T04:32:16.220 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.220 INFO:teuthology.orchestra.run.smithi037.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T04:32:16.220 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T04:32:16.220 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T04:32:16.221 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.221 INFO:teuthology.orchestra.run.smithi037.stdout: # 1m isn't quite enough 2024-09-17T04:32:16.221 INFO:teuthology.orchestra.run.smithi037.stdout: mon_down_mkfs_grace = 2m 2024-09-17T04:32:16.221 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.221 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_filestore_osds = false 2024-09-17T04:32:16.221 INFO:teuthology.orchestra.run.smithi037.stdout: mon down mkfs grace = 300 2024-09-17T04:32:16.221 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:16.221 INFO:teuthology.orchestra.run.smithi037.stdout:[client] 2024-09-17T04:32:16.221 INFO:teuthology.orchestra.run.smithi037.stdout: rgw cache enabled = true 2024-09-17T04:32:16.221 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable ops log = true 2024-09-17T04:32:16.221 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable usage log = true 2024-09-17T04:32:16.222 INFO:teuthology.orchestra.run.smithi037.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T04:32:16.222 INFO:teuthology.orchestra.run.smithi037.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T04:32:16.222 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = "" 2024-09-17T04:32:16.222 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.a] 2024-09-17T04:32:16.222 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.b] 2024-09-17T04:32:16.222 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.c] 2024-09-17T04:32:16.222 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.a] 2024-09-17T04:32:16.222 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.c] 2024-09-17T04:32:16.222 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.b] 2024-09-17T04:32:16.222 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.d] 2024-09-17T04:32:16.223 INFO:teuthology.orchestra.run.smithi092.stdout:[global] 2024-09-17T04:32:16.223 INFO:teuthology.orchestra.run.smithi092.stdout: chdir = "" 2024-09-17T04:32:16.223 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T04:32:16.223 INFO:teuthology.orchestra.run.smithi092.stdout: auth supported = cephx 2024-09-17T04:32:16.223 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.223 INFO:teuthology.orchestra.run.smithi092.stdout: filestore xattr use omap = true 2024-09-17T04:32:16.223 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.223 INFO:teuthology.orchestra.run.smithi092.stdout: mon clock drift allowed = 1.000 2024-09-17T04:32:16.223 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.224 INFO:teuthology.orchestra.run.smithi092.stdout: osd crush chooseleaf type = 0 2024-09-17T04:32:16.224 INFO:teuthology.orchestra.run.smithi092.stdout: auth debug = true 2024-09-17T04:32:16.224 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.224 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on old message = true 2024-09-17T04:32:16.224 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on bug = true 2024-09-17T04:32:16.224 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T04:32:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: mon pg warn max object skew = 0 2024-09-17T04:32:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T04:32:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T04:32:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default size = 2 2024-09-17T04:32:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow primary affinity = true 2024-09-17T04:32:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow pg remap = true 2024-09-17T04:32:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on legacy crush tunables = false 2024-09-17T04:32:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on crush straw calc version zero = false 2024-09-17T04:32:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on no sortbitwise = false 2024-09-17T04:32:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on osd down out interval zero = false 2024-09-17T04:32:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on too few osds = false 2024-09-17T04:32:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T04:32:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T04:32:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: mon_allow_pool_size_one = true 2024-09-17T04:32:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T04:32:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: osd default data pool replay window = 5 2024-09-17T04:32:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: mon allow pool delete = true 2024-09-17T04:32:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: mon cluster log file level = debug 2024-09-17T04:32:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: debug asserts on shutdown = true 2024-09-17T04:32:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: mon health detail to clog = false 2024-09-17T04:32:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: mon host = "172.21.15.37,172.21.15.92,[v2:172.21.15.92:3301,v1:172.21.15.92:6790]" 2024-09-17T04:32:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: fsid = a908a603-430c-4209-8294-0c3460b35f42 2024-09-17T04:32:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.227 INFO:teuthology.orchestra.run.smithi092.stdout:[osd] 2024-09-17T04:32:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: osd journal size = 100 2024-09-17T04:32:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub load threshold = 5.0 2024-09-17T04:32:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub max interval = 600 2024-09-17T04:32:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: osd mclock profile = high_recovery_ops 2024-09-17T04:32:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: osd recover clone overlap = true 2024-09-17T04:32:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: osd recovery max chunk = 1048576 2024-09-17T04:32:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug shutdown = true 2024-09-17T04:32:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug op order = true 2024-09-17T04:32:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug verify stray on activate = true 2024-09-17T04:32:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug trim objects = true 2024-09-17T04:32:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: osd open classes on start = true 2024-09-17T04:32:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug pg log writeout = true 2024-09-17T04:32:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: osd deep scrub update digest min age = 30 2024-09-17T04:32:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: osd map max advance = 10 2024-09-17T04:32:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: journal zero on create = true 2024-09-17T04:32:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: filestore ondisk finisher threads = 3 2024-09-17T04:32:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: filestore apply finisher threads = 3 2024-09-17T04:32:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: bdev debug aio = true 2024-09-17T04:32:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug misdirected ops = true 2024-09-17T04:32:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: bdev async discard = True 2024-09-17T04:32:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: bdev enable discard = True 2024-09-17T04:32:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore allocator = bitmap 2024-09-17T04:32:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore block size = 96636764160 2024-09-17T04:32:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore fsck on mount = True 2024-09-17T04:32:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluefs = 1/20 2024-09-17T04:32:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluestore = 1/20 2024-09-17T04:32:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-09-17T04:32:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: debug osd = 20 2024-09-17T04:32:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: debug rocksdb = 4/10 2024-09-17T04:32:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T04:32:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd full ratio = 0.9 2024-09-17T04:32:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd nearfull ratio = 0.8 2024-09-17T04:32:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: osd failsafe full ratio = 0.95 2024-09-17T04:32:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: osd objectstore = bluestore 2024-09-17T04:32:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.230 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr] 2024-09-17T04:32:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-09-17T04:32:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: debug mgr = 20 2024-09-17T04:32:16.231 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2024-09-17T04:32:16.231 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2024-09-17T04:32:16.231 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2024-09-17T04:32:16.231 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2024-09-17T04:32:16.231 INFO:teuthology.orchestra.run.smithi092.stdout: mgr/telemetry/nag = false 2024-09-17T04:32:16.231 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.231 INFO:teuthology.orchestra.run.smithi092.stdout:[mon] 2024-09-17T04:32:16.231 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-09-17T04:32:16.231 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2024-09-17T04:32:16.231 INFO:teuthology.orchestra.run.smithi092.stdout: debug paxos = 20 2024-09-17T04:32:16.231 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2024-09-17T04:32:16.231 INFO:teuthology.orchestra.run.smithi092.stdout: mon data avail warn = 5 2024-09-17T04:32:16.231 INFO:teuthology.orchestra.run.smithi092.stdout: mon mgr mkfs grace = 240 2024-09-17T04:32:16.231 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2024-09-17T04:32:16.231 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd reporter subtree level = osd 2024-09-17T04:32:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd prime pg temp = true 2024-09-17T04:32:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2024-09-17T04:32:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T04:32:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T04:32:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T04:32:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T04:32:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T04:32:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T04:32:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: # 1m isn't quite enough 2024-09-17T04:32:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: mon_down_mkfs_grace = 2m 2024-09-17T04:32:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.233 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_filestore_osds = false 2024-09-17T04:32:16.233 INFO:teuthology.orchestra.run.smithi092.stdout: mon down mkfs grace = 300 2024-09-17T04:32:16.233 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:32:16.233 INFO:teuthology.orchestra.run.smithi092.stdout:[client] 2024-09-17T04:32:16.233 INFO:teuthology.orchestra.run.smithi092.stdout: rgw cache enabled = true 2024-09-17T04:32:16.233 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable ops log = true 2024-09-17T04:32:16.233 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable usage log = true 2024-09-17T04:32:16.233 INFO:teuthology.orchestra.run.smithi092.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T04:32:16.233 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T04:32:16.233 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = "" 2024-09-17T04:32:16.233 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.a] 2024-09-17T04:32:16.233 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.b] 2024-09-17T04:32:16.233 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.c] 2024-09-17T04:32:16.233 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.a] 2024-09-17T04:32:16.234 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.c] 2024-09-17T04:32:16.234 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.b] 2024-09-17T04:32:16.234 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.d] 2024-09-17T04:32:16.235 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-17T04:32:16.236 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-17T04:32:16.341 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-17T04:32:16.341 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:32:16.341 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-17T04:32:16.358 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:32:16.358 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-17T04:32:16.413 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi037.front.sepia.ceph.com 2024-09-17T04:32:16.414 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:32:16.414 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T04:32:16.414 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T04:32:16.489 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:32:16.489 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:32:16.545 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi092.front.sepia.ceph.com 2024-09-17T04:32:16.545 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:32:16.546 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T04:32:16.546 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T04:32:16.585 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:32:16.585 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:32:16.643 INFO:tasks.ceph:Setting up mon nodes... 2024-09-17T04:32:16.643 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-17T04:32:16.644 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-17T04:32:16.705 INFO:teuthology.orchestra.run.smithi037.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-17T04:32:16.709 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T04:32:16.787 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T04:32:16.790 INFO:tasks.ceph:Setting up mds nodes... 2024-09-17T04:32:16.790 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-17T04:32:16.852 INFO:teuthology.orchestra.run.smithi037.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-17T04:32:16.855 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-17T04:32:16.880 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-17T04:32:16.983 INFO:teuthology.orchestra.run.smithi037.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-17T04:32:16.986 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-17T04:32:17.014 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-17T04:32:17.076 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-17T04:32:17.079 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-17T04:32:17.104 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-17T04:32:17.206 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-17T04:32:17.209 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-17T04:32:17.233 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-17T04:32:17.234 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-17T04:32:17.285 INFO:teuthology.orchestra.run.smithi037.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-17T04:32:17.300 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-17T04:32:17.351 INFO:teuthology.orchestra.run.smithi037.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-17T04:32:17.365 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-17T04:32:17.415 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-17T04:32:17.428 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-17T04:32:17.478 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-17T04:32:17.491 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-17T04:32:17.491 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi037.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-17T04:32:17.491 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-17T04:32:17.516 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T04:32:17.516 INFO:tasks.ceph:role: osd.0 2024-09-17T04:32:17.516 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi037.front.sepia.ceph.com 2024-09-17T04:32:17.516 DEBUG:teuthology.orchestra.run.smithi037:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T04:32:17.589 INFO:teuthology.orchestra.run.smithi037.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T04:32:17.589 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T04:32:17.589 INFO:teuthology.orchestra.run.smithi037.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T04:32:17.589 INFO:teuthology.orchestra.run.smithi037.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T04:32:17.590 INFO:teuthology.orchestra.run.smithi037.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T04:32:17.590 INFO:teuthology.orchestra.run.smithi037.stdout: = sunit=0 swidth=0 blks 2024-09-17T04:32:17.590 INFO:teuthology.orchestra.run.smithi037.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T04:32:17.590 INFO:teuthology.orchestra.run.smithi037.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T04:32:17.590 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T04:32:17.590 INFO:teuthology.orchestra.run.smithi037.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T04:32:18.102 INFO:teuthology.orchestra.run.smithi037.stdout:Discarding blocks...Done. 2024-09-17T04:32:18.103 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi037.front.sepia.ceph.com -o noatime 2024-09-17T04:32:18.104 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-17T04:32:18.260 DEBUG:teuthology.orchestra.run.smithi037:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-17T04:32:18.292 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-17T04:32:18.358 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T04:32:18.358 INFO:tasks.ceph:role: osd.1 2024-09-17T04:32:18.358 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi037.front.sepia.ceph.com 2024-09-17T04:32:18.358 DEBUG:teuthology.orchestra.run.smithi037:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T04:32:18.430 INFO:teuthology.orchestra.run.smithi037.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T04:32:18.431 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T04:32:18.431 INFO:teuthology.orchestra.run.smithi037.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T04:32:18.431 INFO:teuthology.orchestra.run.smithi037.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T04:32:18.431 INFO:teuthology.orchestra.run.smithi037.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T04:32:18.431 INFO:teuthology.orchestra.run.smithi037.stdout: = sunit=0 swidth=0 blks 2024-09-17T04:32:18.431 INFO:teuthology.orchestra.run.smithi037.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T04:32:18.431 INFO:teuthology.orchestra.run.smithi037.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T04:32:18.431 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T04:32:18.431 INFO:teuthology.orchestra.run.smithi037.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T04:32:18.896 INFO:teuthology.orchestra.run.smithi037.stdout:Discarding blocks...Done. 2024-09-17T04:32:18.898 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi037.front.sepia.ceph.com -o noatime 2024-09-17T04:32:18.898 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-17T04:32:19.001 DEBUG:teuthology.orchestra.run.smithi037:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-17T04:32:19.036 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-17T04:32:19.102 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T04:32:19.102 INFO:tasks.ceph:role: osd.2 2024-09-17T04:32:19.102 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi037.front.sepia.ceph.com 2024-09-17T04:32:19.102 DEBUG:teuthology.orchestra.run.smithi037:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T04:32:19.175 INFO:teuthology.orchestra.run.smithi037.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T04:32:19.175 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T04:32:19.175 INFO:teuthology.orchestra.run.smithi037.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T04:32:19.175 INFO:teuthology.orchestra.run.smithi037.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T04:32:19.175 INFO:teuthology.orchestra.run.smithi037.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T04:32:19.175 INFO:teuthology.orchestra.run.smithi037.stdout: = sunit=0 swidth=0 blks 2024-09-17T04:32:19.175 INFO:teuthology.orchestra.run.smithi037.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T04:32:19.175 INFO:teuthology.orchestra.run.smithi037.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T04:32:19.176 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T04:32:19.176 INFO:teuthology.orchestra.run.smithi037.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T04:32:19.653 INFO:teuthology.orchestra.run.smithi037.stdout:Discarding blocks...Done. 2024-09-17T04:32:19.655 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi037.front.sepia.ceph.com -o noatime 2024-09-17T04:32:19.656 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-17T04:32:19.810 DEBUG:teuthology.orchestra.run.smithi037:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-17T04:32:19.843 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-17T04:32:19.908 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T04:32:19.908 INFO:tasks.ceph:role: osd.3 2024-09-17T04:32:19.908 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi037.front.sepia.ceph.com 2024-09-17T04:32:19.909 DEBUG:teuthology.orchestra.run.smithi037:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T04:32:19.981 INFO:teuthology.orchestra.run.smithi037.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T04:32:19.981 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T04:32:19.981 INFO:teuthology.orchestra.run.smithi037.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T04:32:19.982 INFO:teuthology.orchestra.run.smithi037.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T04:32:19.982 INFO:teuthology.orchestra.run.smithi037.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T04:32:19.982 INFO:teuthology.orchestra.run.smithi037.stdout: = sunit=0 swidth=0 blks 2024-09-17T04:32:19.982 INFO:teuthology.orchestra.run.smithi037.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T04:32:19.982 INFO:teuthology.orchestra.run.smithi037.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T04:32:19.982 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T04:32:19.982 INFO:teuthology.orchestra.run.smithi037.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T04:32:20.470 INFO:teuthology.orchestra.run.smithi037.stdout:Discarding blocks...Done. 2024-09-17T04:32:20.472 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi037.front.sepia.ceph.com -o noatime 2024-09-17T04:32:20.472 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-17T04:32:20.607 DEBUG:teuthology.orchestra.run.smithi037:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-17T04:32:20.642 DEBUG:teuthology.orchestra.run.smithi037:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:32:20.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:20.752+0000 7f9a316a5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-17T04:32:20.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:20.753+0000 7f9a316a5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-17T04:32:20.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:20.753+0000 7f9a316a5640 -1 bdev(0x563b0d65d900 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-17T04:32:20.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:20.753+0000 7f9a316a5640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-17T04:32:23.910 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-17T04:32:23.941 DEBUG:teuthology.orchestra.run.smithi037:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:32:24.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:24.046+0000 7f66bbe0e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-17T04:32:24.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:24.047+0000 7f66bbe0e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-17T04:32:24.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:24.047+0000 7f66bbe0e640 -1 bdev(0x55ddb5e33900 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-17T04:32:24.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:24.047+0000 7f66bbe0e640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-17T04:32:27.457 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-17T04:32:27.486 DEBUG:teuthology.orchestra.run.smithi037:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:32:27.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:27.589+0000 7fc7fea13640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-17T04:32:27.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:27.589+0000 7fc7fea13640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-17T04:32:27.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:27.589+0000 7fc7fea13640 -1 bdev(0x561fecca9900 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-17T04:32:27.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:27.589+0000 7fc7fea13640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-17T04:32:30.513 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-17T04:32:30.545 DEBUG:teuthology.orchestra.run.smithi037:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:32:30.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:30.650+0000 7f0f7400f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-17T04:32:30.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:30.651+0000 7f0f7400f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-17T04:32:30.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:30.651+0000 7f0f7400f640 -1 bdev(0x556f45c7d900 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-17T04:32:30.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:30.651+0000 7f0f7400f640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-17T04:32:33.817 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-17T04:32:33.848 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-17T04:32:33.878 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T04:32:33.878 INFO:tasks.ceph:role: osd.4 2024-09-17T04:32:33.878 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-17T04:32:33.878 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T04:32:33.953 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T04:32:33.953 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T04:32:33.953 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T04:32:33.953 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T04:32:33.953 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T04:32:33.953 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-09-17T04:32:33.954 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T04:32:33.954 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T04:32:33.954 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T04:32:33.954 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T04:32:34.475 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-09-17T04:32:34.478 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-09-17T04:32:34.478 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-17T04:32:34.632 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-17T04:32:34.662 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-17T04:32:34.728 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T04:32:34.728 INFO:tasks.ceph:role: osd.5 2024-09-17T04:32:34.728 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-17T04:32:34.728 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T04:32:34.800 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T04:32:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T04:32:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T04:32:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T04:32:34.801 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T04:32:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-09-17T04:32:34.801 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T04:32:34.801 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T04:32:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T04:32:34.801 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T04:32:35.250 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-09-17T04:32:35.253 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-09-17T04:32:35.253 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-17T04:32:35.337 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-17T04:32:35.407 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-17T04:32:35.472 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T04:32:35.472 INFO:tasks.ceph:role: osd.6 2024-09-17T04:32:35.472 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-17T04:32:35.472 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T04:32:35.544 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T04:32:35.544 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T04:32:35.544 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T04:32:35.544 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T04:32:35.544 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T04:32:35.544 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-09-17T04:32:35.544 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T04:32:35.545 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T04:32:35.545 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T04:32:35.545 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T04:32:36.036 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-09-17T04:32:36.037 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-09-17T04:32:36.037 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-17T04:32:36.177 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-17T04:32:36.208 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-17T04:32:36.273 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T04:32:36.273 INFO:tasks.ceph:role: osd.7 2024-09-17T04:32:36.274 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-17T04:32:36.274 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T04:32:36.345 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T04:32:36.345 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T04:32:36.346 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T04:32:36.346 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T04:32:36.346 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T04:32:36.346 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-09-17T04:32:36.346 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T04:32:36.346 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T04:32:36.346 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T04:32:36.346 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T04:32:36.858 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-09-17T04:32:36.860 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-09-17T04:32:36.860 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-17T04:32:36.983 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-17T04:32:37.016 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:32:37.126 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:37.125+0000 7f39d820b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-17T04:32:37.126 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:37.125+0000 7f39d820b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-17T04:32:37.126 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:37.125+0000 7f39d820b640 -1 bdev(0x5574f42d9900 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-17T04:32:37.126 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:37.125+0000 7f39d820b640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-17T04:32:40.537 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-17T04:32:40.566 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:32:40.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:40.668+0000 7f6e9dddd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-17T04:32:40.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:40.669+0000 7f6e9dddd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-17T04:32:40.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:40.669+0000 7f6e9dddd640 -1 bdev(0x561792f03900 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-17T04:32:40.669 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:40.669+0000 7f6e9dddd640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-17T04:32:44.077 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-17T04:32:44.104 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:32:44.183 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:32:44.187 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:32:44.204 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:44.204+0000 7f8b4ee11640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-17T04:32:44.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:44.204+0000 7f8b4ee11640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-17T04:32:44.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:44.204+0000 7f8b4ee11640 -1 bdev(0x5646133f3900 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-17T04:32:44.205 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:44.205+0000 7f8b4ee11640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-17T04:32:47.364 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-17T04:32:47.393 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:32:47.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:47.499+0000 7f62e1010640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-17T04:32:47.499 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:47.499+0000 7f62e1010640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-17T04:32:47.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:47.500+0000 7f62e1010640 -1 bdev(0x56283022b900 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-17T04:32:47.500 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:47.500+0000 7f62e1010640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-17T04:32:50.899 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-17T04:32:50.927 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-17T04:32:50.927 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:32:50.927 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-17T04:32:50.960 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:32:50.960 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-17T04:32:51.032 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:32:51.032 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-17T04:32:51.103 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:32:51.104 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-17T04:32:51.174 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:32:51.174 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-17T04:32:51.245 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:32:51.246 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-17T04:32:51.317 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:32:51.317 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-17T04:32:51.387 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:32:51.387 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-17T04:32:51.415 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:32:51.416 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-17T04:32:51.483 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:32:51.483 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-17T04:32:51.550 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:32:51.550 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-17T04:32:51.616 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:32:51.616 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-17T04:32:51.683 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:32:51.683 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-17T04:32:51.749 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:32:51.749 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-17T04:32:51.815 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:32:51.815 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-17T04:32:51.834 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:32:51.835 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-17T04:32:51.893 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:32:51.893 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-17T04:32:51.909 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:32:51.909 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-17T04:32:51.966 INFO:tasks.ceph:Adding keys to all mons... 2024-09-17T04:32:51.966 DEBUG:teuthology.orchestra.run.smithi037:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T04:32:51.968 DEBUG:teuthology.orchestra.run.smithi092:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T04:32:52.010 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr.y] 2024-09-17T04:32:52.010 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBQBulmow4mKhAAhECLmAc2tfC/HhfQsOGxeQ== 2024-09-17T04:32:52.010 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.a] 2024-09-17T04:32:52.010 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBQBulmh1PhMhAALoHZDSZgGMyaEd8e4M+2KA== 2024-09-17T04:32:52.010 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.c] 2024-09-17T04:32:52.010 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBQBulmjaCsOhAA3c9mpeC0D8K9utnc6/13zw== 2024-09-17T04:32:52.010 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.0] 2024-09-17T04:32:52.010 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBUBulm51ntLBAA0Hm4yP85dCj3Ee10jnYVjw== 2024-09-17T04:32:52.010 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.1] 2024-09-17T04:32:52.010 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBYBulmYkPXAhAA5+ybQVYsBLT2FkgWOzoxNA== 2024-09-17T04:32:52.010 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.2] 2024-09-17T04:32:52.011 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBbBulmcpcoIxAAFy7BHRdyJ7X+bCpf/LYOLg== 2024-09-17T04:32:52.011 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.3] 2024-09-17T04:32:52.011 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBeBulmYt3XJhAA9N1FTXul2rk6U1bkQ5VraQ== 2024-09-17T04:32:52.011 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr.x] 2024-09-17T04:32:52.011 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBQBulmL235LhAAMvjhUWzQszGESkHd4NlKYQ== 2024-09-17T04:32:52.011 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.b] 2024-09-17T04:32:52.011 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBRBulmsymlBBAAgBYGOXNn7aSEHMxkvsewJQ== 2024-09-17T04:32:52.011 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.d] 2024-09-17T04:32:52.011 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBRBulmnIlcDBAAXbqz9EnlFmxAb2E8OO9/KQ== 2024-09-17T04:32:52.011 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.4] 2024-09-17T04:32:52.011 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBlBulm4E2QBxAARmAT4UEHND01x5vb8fHb0w== 2024-09-17T04:32:52.011 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.5] 2024-09-17T04:32:52.012 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBoBulm6BPtJxAArpGCRHadXl1WNGsyM19FUg== 2024-09-17T04:32:52.012 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.6] 2024-09-17T04:32:52.012 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBsBulmBt8/DBAAk587ed/x3qfWy9b0iPYSqA== 2024-09-17T04:32:52.012 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.7] 2024-09-17T04:32:52.012 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBvBulmLSjWHRAA7crk7TbyBJ1ihfqGMd/G1w== 2024-09-17T04:32:52.012 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-09-17T04:32:52.012 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBRBulm3vQYERAA/gAxWQECQvkbphM8KrWgUQ== 2024-09-17T04:32:52.012 INFO:teuthology.orchestra.run.smithi037.stdout:[client.2] 2024-09-17T04:32:52.012 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBRBulmQnAHFRAAujq2Mnektuzbv+nd8v+CTg== 2024-09-17T04:32:52.012 INFO:teuthology.orchestra.run.smithi037.stdout:[client.1] 2024-09-17T04:32:52.012 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBRBulmh13QGBAAhN/ly2QhbEKp3oAclmXC4w== 2024-09-17T04:32:52.012 INFO:teuthology.orchestra.run.smithi037.stdout:[client.3] 2024-09-17T04:32:52.012 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBRBulmdqWRHBAAE91kKLf4sVqvfbS2oxFHfg== 2024-09-17T04:32:52.030 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.y] 2024-09-17T04:32:52.030 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBQBulmow4mKhAAhECLmAc2tfC/HhfQsOGxeQ== 2024-09-17T04:32:52.030 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.a] 2024-09-17T04:32:52.030 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBQBulmh1PhMhAALoHZDSZgGMyaEd8e4M+2KA== 2024-09-17T04:32:52.030 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.c] 2024-09-17T04:32:52.030 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBQBulmjaCsOhAA3c9mpeC0D8K9utnc6/13zw== 2024-09-17T04:32:52.031 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.0] 2024-09-17T04:32:52.031 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBUBulm51ntLBAA0Hm4yP85dCj3Ee10jnYVjw== 2024-09-17T04:32:52.031 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.1] 2024-09-17T04:32:52.031 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBYBulmYkPXAhAA5+ybQVYsBLT2FkgWOzoxNA== 2024-09-17T04:32:52.031 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.2] 2024-09-17T04:32:52.031 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBbBulmcpcoIxAAFy7BHRdyJ7X+bCpf/LYOLg== 2024-09-17T04:32:52.031 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.3] 2024-09-17T04:32:52.031 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBeBulmYt3XJhAA9N1FTXul2rk6U1bkQ5VraQ== 2024-09-17T04:32:52.031 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.x] 2024-09-17T04:32:52.031 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBQBulmL235LhAAMvjhUWzQszGESkHd4NlKYQ== 2024-09-17T04:32:52.031 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.b] 2024-09-17T04:32:52.031 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBRBulmsymlBBAAgBYGOXNn7aSEHMxkvsewJQ== 2024-09-17T04:32:52.031 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.d] 2024-09-17T04:32:52.031 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBRBulmnIlcDBAAXbqz9EnlFmxAb2E8OO9/KQ== 2024-09-17T04:32:52.032 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.4] 2024-09-17T04:32:52.032 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBlBulm4E2QBxAARmAT4UEHND01x5vb8fHb0w== 2024-09-17T04:32:52.032 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.5] 2024-09-17T04:32:52.032 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBoBulm6BPtJxAArpGCRHadXl1WNGsyM19FUg== 2024-09-17T04:32:52.032 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.6] 2024-09-17T04:32:52.032 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBsBulmBt8/DBAAk587ed/x3qfWy9b0iPYSqA== 2024-09-17T04:32:52.032 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.7] 2024-09-17T04:32:52.032 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBvBulmLSjWHRAA7crk7TbyBJ1ihfqGMd/G1w== 2024-09-17T04:32:52.032 INFO:teuthology.orchestra.run.smithi092.stdout:[client.0] 2024-09-17T04:32:52.032 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBRBulm3vQYERAA/gAxWQECQvkbphM8KrWgUQ== 2024-09-17T04:32:52.032 INFO:teuthology.orchestra.run.smithi092.stdout:[client.2] 2024-09-17T04:32:52.032 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBRBulmQnAHFRAAujq2Mnektuzbv+nd8v+CTg== 2024-09-17T04:32:52.032 INFO:teuthology.orchestra.run.smithi092.stdout:[client.1] 2024-09-17T04:32:52.032 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBRBulmh13QGBAAhN/ly2QhbEKp3oAclmXC4w== 2024-09-17T04:32:52.033 INFO:teuthology.orchestra.run.smithi092.stdout:[client.3] 2024-09-17T04:32:52.033 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBRBulmdqWRHBAAE91kKLf4sVqvfbS2oxFHfg== 2024-09-17T04:32:52.033 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T04:32:52.056 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T04:32:52.135 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T04:32:52.138 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T04:32:52.205 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T04:32:52.207 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T04:32:52.272 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:32:52.274 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:32:52.337 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:32:52.339 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:32:52.400 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:32:52.403 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:32:52.464 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:32:52.466 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:32:52.528 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T04:32:52.530 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T04:32:52.592 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T04:32:52.595 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T04:32:52.659 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T04:32:52.661 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T04:32:52.727 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:32:52.729 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:32:52.790 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:32:52.793 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:32:52.855 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:32:52.857 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:32:52.918 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:32:52.921 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T04:32:52.982 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T04:32:52.984 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T04:32:53.049 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T04:32:53.051 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T04:32:53.116 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T04:32:53.119 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T04:32:53.184 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T04:32:53.186 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T04:32:53.253 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-17T04:32:53.253 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-17T04:32:53.279 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T04:32:53.386 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-17T04:32:53.413 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-17T04:32:53.439 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T04:32:53.546 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-17T04:32:53.571 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-17T04:32:53.636 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T04:32:53.743 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-17T04:32:53.767 DEBUG:teuthology.orchestra.run.smithi037:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:32:53.769 DEBUG:teuthology.orchestra.run.smithi092:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T04:32:53.823 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-17T04:32:53.823 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-17T04:32:53.823 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-17T04:32:53.826 INFO:tasks.ceph.mon.a:Started 2024-09-17T04:32:53.826 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-17T04:32:53.826 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-17T04:32:53.865 INFO:tasks.ceph.mon.b:Started 2024-09-17T04:32:53.865 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-17T04:32:53.865 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-17T04:32:53.867 INFO:tasks.ceph.mon.c:Started 2024-09-17T04:32:53.868 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-17T04:32:53.868 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-17T04:32:53.868 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-17T04:32:53.870 INFO:tasks.ceph.mgr.y:Started 2024-09-17T04:32:53.870 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-17T04:32:53.870 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-17T04:32:53.874 INFO:tasks.ceph.mgr.x:Started 2024-09-17T04:32:53.874 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-17T04:32:53.875 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-17T04:32:53.877 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-17T04:32:53.877 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-17T04:32:53.920 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-17T04:32:53.920 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-17T04:32:53.962 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-17T04:32:53.963 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-17T04:32:54.006 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-17T04:32:54.007 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-17T04:32:54.050 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-17T04:32:54.051 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-17T04:32:54.093 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-17T04:32:54.093 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-17T04:32:54.137 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-17T04:32:54.137 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-17T04:32:54.180 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-17T04:32:54.180 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-17T04:32:54.223 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-17T04:32:54.224 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-17T04:32:54.268 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-17T04:32:54.268 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-17T04:32:54.311 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-17T04:32:54.311 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-17T04:32:54.355 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-17T04:32:54.356 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-17T04:32:54.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.372+0000 7fe07affd640 1 -- 172.21.15.37:0/1133744360 <== mon.2 v2:172.21.15.92:3301/0 10 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v4)=0 v4) ==== 109+0+0 (secure 0 0 0) 0x7fe07c01f030 con 0x7fe074004dd0 2024-09-17T04:32:54.373 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-17T04:32:54.373 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-17T04:32:54.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.376+0000 7f98f21b0640 1 -- 172.21.15.37:0/879007695 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f98c4005310 con 0x7f98d8004850 2024-09-17T04:32:54.382 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-17T04:32:54.382 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-17T04:32:54.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.385+0000 7f98e8ff9640 1 -- 172.21.15.37:0/879007695 <== mon.0 v2:172.21.15.37:3300/0 10 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v5)=0 v5) ==== 123+0+0 (secure 0 0 0) 0x7f98ec0b50a0 con 0x7f98d8004850 2024-09-17T04:32:54.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.386+0000 7f98e8ff9640 1 -- 172.21.15.37:0/879007695 <== mon.0 v2:172.21.15.37:3300/0 11 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f98dc029020 con 0x7f98d8004850 2024-09-17T04:32:54.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.383+0000 7fe07affd640 1 -- 172.21.15.37:0/1133744360 <== mon.2 v2:172.21.15.92:3301/0 11 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7fe07c01a7a0 con 0x7fe074004dd0 2024-09-17T04:32:54.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.383+0000 7fe090e4d640 1 -- 172.21.15.37:0/1133744360 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe074004dd0 msgr2=0x7fe08c09eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.383+0000 7fe090e4d640 1 --2- 172.21.15.37:0/1133744360 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe074004dd0 0x7fe08c09eba0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00da10 tx=0x7fe07c00ded0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.384+0000 7fe090e4d640 1 -- 172.21.15.37:0/1133744360 shutdown_connections 2024-09-17T04:32:54.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.384+0000 7fe090e4d640 1 --2- 172.21.15.37:0/1133744360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe08c0ae560 0x7fe08c0afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.384+0000 7fe090e4d640 1 --2- 172.21.15.37:0/1133744360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe08c13ea80 0x7fe08c0ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.384+0000 7fe090e4d640 1 --2- 172.21.15.37:0/1133744360 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe074004dd0 0x7fe08c09eba0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.384+0000 7fe090e4d640 1 -- 172.21.15.37:0/1133744360 >> 172.21.15.37:0/1133744360 conn(0x7fe08c0b65d0 msgr2=0x7fe08c0b41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:54.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.385+0000 7fe090e4d640 1 -- 172.21.15.37:0/1133744360 shutdown_connections 2024-09-17T04:32:54.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.383+0000 7fa8cbfff640 1 -- 172.21.15.37:0/2499072406 <== mon.2 v2:172.21.15.92:3301/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7fa8d400f3e0 con 0x7fa8dc052c00 2024-09-17T04:32:54.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.385+0000 7f326f7fe640 1 -- 172.21.15.37:0/953593744 <== mon.1 v2:172.21.15.92:3300/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f32780557b0 con 0x7f32800550b0 2024-09-17T04:32:54.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.385+0000 7fe090e4d640 1 -- 172.21.15.37:0/1133744360 wait complete. 2024-09-17T04:32:54.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.389+0000 7f98e1ffb640 1 -- 172.21.15.37:0/879007695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98d8004850 msgr2=0x7f98ec1f4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.389+0000 7f98e1ffb640 1 --2- 172.21.15.37:0/879007695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98d8004850 0x7f98ec1f4680 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f98dc001050 tx=0x7f98dc00dd80 comp rx=0 tx=0).stop 2024-09-17T04:32:54.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.389+0000 7f98e1ffb640 1 -- 172.21.15.37:0/879007695 shutdown_connections 2024-09-17T04:32:54.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.389+0000 7f98e1ffb640 1 --2- 172.21.15.37:0/879007695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98ec0b4750 0x7f98ec0b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.389+0000 7f98e1ffb640 1 --2- 172.21.15.37:0/879007695 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f98ec154350 0x7f98ec152da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.389+0000 7f98e1ffb640 1 --2- 172.21.15.37:0/879007695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98d8004850 0x7f98ec1f4680 secure :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f98dc001050 tx=0x7f98dc00dd80 comp rx=0 tx=0).stop 2024-09-17T04:32:54.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.389+0000 7f98e1ffb640 1 -- 172.21.15.37:0/879007695 >> 172.21.15.37:0/879007695 conn(0x7f98ec0b2790 msgr2=0x7f98ec0cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:54.392 INFO:tasks.ceph:setting config [mgr] client_mount_timeout = 30 2024-09-17T04:32:54.392 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mgr client_mount_timeout 30 2024-09-17T04:32:54.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.394+0000 7f98e1ffb640 1 -- 172.21.15.37:0/879007695 shutdown_connections 2024-09-17T04:32:54.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.394+0000 7f98e1ffb640 1 -- 172.21.15.37:0/879007695 wait complete. 2024-09-17T04:32:54.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.405+0000 7faad6ffd640 1 -- 172.21.15.37:0/4036374517 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad8036c40 con 0x7faae015cc40 2024-09-17T04:32:54.406 INFO:tasks.ceph:setting config [mgr] debug_client = 20 2024-09-17T04:32:54.407 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mgr debug_client 20 2024-09-17T04:32:54.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.405+0000 7faae850b640 1 -- 172.21.15.37:0/4036374517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae02029b0 con 0x7faae015cc40 2024-09-17T04:32:54.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.405+0000 7faae850b640 1 -- 172.21.15.37:0/4036374517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae01556a0 con 0x7faae015cc40 2024-09-17T04:32:54.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.406+0000 7faad4ff9640 1 -- 172.21.15.37:0/4036374517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faae015ac50 con 0x7faae015cc40 2024-09-17T04:32:54.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.409+0000 7faad6ffd640 1 -- 172.21.15.37:0/4036374517 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faad80407c0 con 0x7faae015cc40 2024-09-17T04:32:54.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.409+0000 7faad6ffd640 1 -- 172.21.15.37:0/4036374517 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7faad8048770 con 0x7faae015cc40 2024-09-17T04:32:54.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.415+0000 7faad6ffd640 1 -- 172.21.15.37:0/4036374517 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7faad804d020 con 0x7faae015cc40 2024-09-17T04:32:54.416 INFO:tasks.ceph:setting config [mgr] debug_mgr = 20 2024-09-17T04:32:54.416 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mgr debug_mgr 20 2024-09-17T04:32:54.422 INFO:tasks.ceph:setting config [mgr] debug_ms = 1 2024-09-17T04:32:54.422 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mgr debug_ms 1 2024-09-17T04:32:54.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.427+0000 7fa8e3fff640 1 -- 172.21.15.37:0/2499072406 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fa8bc005310 con 0x7fa8dc052c00 2024-09-17T04:32:54.430 INFO:tasks.ceph:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-17T04:32:54.430 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mgr mon_warn_on_pool_no_app False 2024-09-17T04:32:54.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.431+0000 7fce53569640 1 -- 172.21.15.37:0/3363170167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce440df1b0 msgr2=0x7fce44005ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.431+0000 7fce53569640 1 --2- 172.21.15.37:0/3363170167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce440df1b0 0x7fce44005ce0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009830 tx=0x7fce3c02a5b0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.432+0000 7fce53569640 1 -- 172.21.15.37:0/3363170167 shutdown_connections 2024-09-17T04:32:54.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.433+0000 7fce53569640 1 --2- 172.21.15.37:0/3363170167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce440df1b0 0x7fce44005ce0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.433+0000 7fce53569640 1 -- 172.21.15.37:0/3363170167 >> 172.21.15.37:0/3363170167 conn(0x7fce44051450 msgr2=0x7fce44051890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:54.439 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-17T04:32:54.439 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-17T04:32:54.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.437+0000 7fa8cbfff640 1 -- 172.21.15.37:0/2499072406 <== mon.2 v2:172.21.15.92:3301/0 11 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8d400f780 con 0x7fa8dc052c00 2024-09-17T04:32:54.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.437+0000 7fa8cbfff640 1 -- 172.21.15.37:0/2499072406 <== mon.2 v2:172.21.15.92:3301/0 12 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7fa8d400fb70 con 0x7fa8dc052c00 2024-09-17T04:32:54.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.439+0000 7fa8e3fff640 1 -- 172.21.15.37:0/2499072406 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa8dc052c00 msgr2=0x7fa8dc17d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.439+0000 7fa8e3fff640 1 --2- 172.21.15.37:0/2499072406 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa8dc052c00 0x7fa8dc17d780 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40010e0 tx=0x7fa8d40015a0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.439+0000 7fa8e3fff640 1 -- 172.21.15.37:0/2499072406 shutdown_connections 2024-09-17T04:32:54.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.439+0000 7fa8e3fff640 1 --2- 172.21.15.37:0/2499072406 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa8dc052c00 0x7fa8dc17d780 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.439+0000 7fa8e3fff640 1 --2- 172.21.15.37:0/2499072406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8dc0370f0 0x7fa8dc17d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.439+0000 7fa8e3fff640 1 --2- 172.21.15.37:0/2499072406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8dc0351b0 0x7fa8dc0d5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.439+0000 7fa8e3fff640 1 -- 172.21.15.37:0/2499072406 >> 172.21.15.37:0/2499072406 conn(0x7fa8dc01eac0 msgr2=0x7fa8dc005a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:54.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.436+0000 7f326f7fe640 1 -- 172.21.15.37:0/953593744 <== mon.1 v2:172.21.15.92:3300/0 11 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3278055ba0 con 0x7f32800550b0 2024-09-17T04:32:54.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.442+0000 7fa8e3fff640 1 -- 172.21.15.37:0/2499072406 shutdown_connections 2024-09-17T04:32:54.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.442+0000 7fa8e3fff640 1 -- 172.21.15.37:0/2499072406 wait complete. 2024-09-17T04:32:54.448 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-17T04:32:54.448 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-17T04:32:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.439+0000 7faad6ffd640 1 -- 172.21.15.37:0/4036374517 <== mon.0 v2:172.21.15.37:3300/0 7 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faad8048b30 con 0x7faae015cc40 2024-09-17T04:32:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.445+0000 7fce53569640 1 -- 172.21.15.37:0/3363170167 shutdown_connections 2024-09-17T04:32:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.445+0000 7fce53569640 1 -- 172.21.15.37:0/3363170167 wait complete. 2024-09-17T04:32:54.461 DEBUG:tasks.ceph:set 22 configs 2024-09-17T04:32:54.461 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi037') 2024-09-17T04:32:54.462 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi037') 2024-09-17T04:32:54.462 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi037') 2024-09-17T04:32:54.462 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi037') 2024-09-17T04:32:54.462 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi037') 2024-09-17T04:32:54.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.460+0000 7fce53569640 1 Processor -- start 2024-09-17T04:32:54.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.460+0000 7fce53569640 1 -- start start 2024-09-17T04:32:54.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.469+0000 7fce53569640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce44009bd0 0x7fce440dd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.478+0000 7fce53569640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce440df1b0 0x7fce44178930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.478+0000 7fce53569640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce44178e70 0x7fce4401e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.478+0000 7fce51b06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce44178e70 0x7fce4401e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.478+0000 7fce51b06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce44178e70 0x7fce4401e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56476/0 (socket says 172.21.15.37:56476) 2024-09-17T04:32:54.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.478+0000 7fce51b06640 1 -- 172.21.15.37:0/2837040995 learned_addr learned my addr 172.21.15.37:0/2837040995 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:32:54.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.478+0000 7fce53569640 1 -- 172.21.15.37:0/2837040995 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fce44006df0 con 0x7fce44178e70 2024-09-17T04:32:54.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.478+0000 7fce53569640 1 -- 172.21.15.37:0/2837040995 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fce4400be70 con 0x7fce44009bd0 2024-09-17T04:32:54.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.478+0000 7fce53569640 1 -- 172.21.15.37:0/2837040995 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fce44006f70 con 0x7fce440df1b0 2024-09-17T04:32:54.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.479+0000 7fce51305640 1 --2- 172.21.15.37:0/2837040995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce44009bd0 0x7fce440dd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.479+0000 7fce50b04640 1 --2- 172.21.15.37:0/2837040995 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce440df1b0 0x7fce44178930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.479+0000 7fce51b06640 1 -- 172.21.15.37:0/2837040995 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce440df1b0 msgr2=0x7fce44178930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.479+0000 7fce51b06640 1 --2- 172.21.15.37:0/2837040995 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce440df1b0 0x7fce44178930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.479+0000 7fce51b06640 1 -- 172.21.15.37:0/2837040995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce44009bd0 msgr2=0x7fce440dd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.479+0000 7fce51b06640 1 --2- 172.21.15.37:0/2837040995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce44009bd0 0x7fce440dd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.479+0000 7fce51b06640 1 -- 172.21.15.37:0/2837040995 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce44179280 con 0x7fce44178e70 2024-09-17T04:32:54.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.480+0000 7fce51b06640 1 --2- 172.21.15.37:0/2837040995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce44178e70 0x7fce4401e440 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009920 tx=0x7fce3c033ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:54.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.481+0000 7fce51305640 1 --2- 172.21.15.37:0/2837040995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce44009bd0 0x7fce440dd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:54.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.481+0000 7fce41ffb640 1 -- 172.21.15.37:0/2837040995 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c044070 con 0x7fce44178e70 2024-09-17T04:32:54.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.481+0000 7fce41ffb640 1 -- 172.21.15.37:0/2837040995 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce3c03c270 con 0x7fce44178e70 2024-09-17T04:32:54.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.481+0000 7fce53569640 1 -- 172.21.15.37:0/2837040995 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce44032710 con 0x7fce44178e70 2024-09-17T04:32:54.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.482+0000 7fce53569640 1 -- 172.21.15.37:0/2837040995 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce4402b660 con 0x7fce44178e70 2024-09-17T04:32:54.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.484+0000 7fce41ffb640 1 -- 172.21.15.37:0/2837040995 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c040c20 con 0x7fce44178e70 2024-09-17T04:32:54.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.484+0000 7fce41ffb640 1 -- 172.21.15.37:0/2837040995 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fce3c047070 con 0x7fce44178e70 2024-09-17T04:32:54.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.484+0000 7fce41ffb640 1 -- 172.21.15.37:0/2837040995 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fce3c0035f0 con 0x7fce44178e70 2024-09-17T04:32:54.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.484+0000 7fce377fe640 1 -- 172.21.15.37:0/2837040995 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce440073d0 con 0x7fce44178e70 2024-09-17T04:32:54.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.490+0000 7fce41ffb640 1 -- 172.21.15.37:0/2837040995 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fce44179280 con 0x7fce44178e70 2024-09-17T04:32:54.500 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi037') 2024-09-17T04:32:54.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.501+0000 7f8ccd539640 1 -- 172.21.15.37:0/4026544492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cb0004800 msgr2=0x7f8cb0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.501+0000 7f8ccd539640 1 --2- 172.21.15.37:0/4026544492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cb0004800 0x7f8cb0004c50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8004170 tx=0x7f8cb0005830 comp rx=0 tx=0).stop 2024-09-17T04:32:54.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.501+0000 7f8ccd539640 1 -- 172.21.15.37:0/4026544492 shutdown_connections 2024-09-17T04:32:54.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.501+0000 7f8ccd539640 1 --2- 172.21.15.37:0/4026544492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cb0004800 0x7f8cb0004c50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.501+0000 7f8ccd539640 1 -- 172.21.15.37:0/4026544492 >> 172.21.15.37:0/4026544492 conn(0x7f8cc001d9d0 msgr2=0x7f8cc0039690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:54.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.505+0000 7f8ccd539640 1 -- 172.21.15.37:0/4026544492 shutdown_connections 2024-09-17T04:32:54.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.505+0000 7f8ccd539640 1 -- 172.21.15.37:0/4026544492 wait complete. 2024-09-17T04:32:54.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.508+0000 7f8ccd539640 1 Processor -- start 2024-09-17T04:32:54.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.508+0000 7f8ccd539640 1 -- start start 2024-09-17T04:32:54.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.511+0000 7f8ccd539640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8cb0004800 0x7f8cc00320f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.511+0000 7f8ccd539640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cc00e1640 0x7f8cc00e1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.511+0000 7f8ccd539640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cc00d7f00 0x7f8cc00d8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.511+0000 7f8ccd539640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc00e6e80 con 0x7f8cc00e1640 2024-09-17T04:32:54.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.511+0000 7f8cc67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cc00e1640 0x7f8cc00e1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.512+0000 7f8cc67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cc00e1640 0x7f8cc00e1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56508/0 (socket says 172.21.15.37:56508) 2024-09-17T04:32:54.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.512+0000 7f8cc67fc640 1 -- 172.21.15.37:0/4077293742 learned_addr learned my addr 172.21.15.37:0/4077293742 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:32:54.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.512+0000 7f8ccd539640 1 -- 172.21.15.37:0/4077293742 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc00e6d00 con 0x7f8cc00d7f00 2024-09-17T04:32:54.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.512+0000 7f8ccd539640 1 -- 172.21.15.37:0/4077293742 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f8cc0033ef0 con 0x7f8cb0004800 2024-09-17T04:32:54.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.513+0000 7f8cc67fc640 1 -- 172.21.15.37:0/4077293742 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8cb0004800 msgr2=0x7f8cc00320f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T04:32:54.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.513+0000 7f8cc67fc640 1 --2- 172.21.15.37:0/4077293742 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8cb0004800 0x7f8cc00320f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.513+0000 7f8cc67fc640 1 -- 172.21.15.37:0/4077293742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cc00d7f00 msgr2=0x7f8cc00d8310 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T04:32:54.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.513+0000 7f8cc67fc640 1 --2- 172.21.15.37:0/4077293742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cc00d7f00 0x7f8cc00d8310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.513+0000 7f8cc67fc640 1 -- 172.21.15.37:0/4077293742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc01868a0 con 0x7f8cc00e1640 2024-09-17T04:32:54.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.513+0000 7f8cc67fc640 1 --2- 172.21.15.37:0/4077293742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cc00e1640 0x7f8cc00e1a50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8004170 tx=0x7f8cb8003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:54.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.514+0000 7f8caf7fe640 1 -- 172.21.15.37:0/4077293742 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8007140 con 0x7f8cc00e1640 2024-09-17T04:32:54.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.514+0000 7f8ccd539640 1 -- 172.21.15.37:0/4077293742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cc00e1f90 con 0x7f8cc00e1640 2024-09-17T04:32:54.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.515+0000 7f8ccd539640 1 -- 172.21.15.37:0/4077293742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cc0033600 con 0x7f8cc00e1640 2024-09-17T04:32:54.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.517+0000 7f8caf7fe640 1 -- 172.21.15.37:0/4077293742 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8cb80072e0 con 0x7f8cc00e1640 2024-09-17T04:32:54.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.525+0000 7f8caf7fe640 1 -- 172.21.15.37:0/4077293742 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8004980 con 0x7f8cc00e1640 2024-09-17T04:32:54.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.526+0000 7f8caf7fe640 1 -- 172.21.15.37:0/4077293742 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8cb8004b60 con 0x7f8cc00e1640 2024-09-17T04:32:54.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.533+0000 7f8caf7fe640 1 -- 172.21.15.37:0/4077293742 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8cb8034da0 con 0x7f8cc00e1640 2024-09-17T04:32:54.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.534+0000 7f8cad7fa640 1 -- 172.21.15.37:0/4077293742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca8005290 con 0x7f8cc00e1640 2024-09-17T04:32:54.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.538+0000 7f8caf7fe640 1 -- 172.21.15.37:0/4077293742 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f8cc00e1f90 con 0x7f8cc00e1640 2024-09-17T04:32:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.693+0000 7f326d7fa640 1 -- 172.21.15.37:0/953593744 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f32600052f0 con 0x7f32800550b0 2024-09-17T04:32:54.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.702+0000 7faad4ff9640 1 -- 172.21.15.37:0/4036374517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7faae006f3c0 con 0x7faae015cc40 2024-09-17T04:32:54.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.708+0000 7f326f7fe640 1 -- 172.21.15.37:0/953593744 <== mon.1 v2:172.21.15.92:3300/0 12 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v7)=0 v7) ==== 108+0+0 (secure 0 0 0) 0x7f3278035070 con 0x7f32800550b0 2024-09-17T04:32:54.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.708+0000 7ff9ba69b640 1 -- 172.21.15.37:0/1940827195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9ac0e33a0 msgr2=0x7ff9ac051ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.708+0000 7ff9ba69b640 1 --2- 172.21.15.37:0/1940827195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9ac0e33a0 0x7ff9ac051ff0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff9b407ab50 tx=0x7ff9b40a6490 comp rx=0 tx=0).stop 2024-09-17T04:32:54.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.709+0000 7ff9ba69b640 1 -- 172.21.15.37:0/1940827195 shutdown_connections 2024-09-17T04:32:54.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.709+0000 7ff9ba69b640 1 --2- 172.21.15.37:0/1940827195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9ac0e33a0 0x7ff9ac051ff0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.709+0000 7ff9ba69b640 1 --2- 172.21.15.37:0/1940827195 >> v2:172.21.15.92:3300/0 conn(0x7ff9ac006070 0x7ff9ac03a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.709+0000 7ff9ba69b640 1 -- 172.21.15.37:0/1940827195 >> 172.21.15.37:0/1940827195 conn(0x7ff9ac01e230 msgr2=0x7ff9ac039ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:54.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.709+0000 7f326d7fa640 1 -- 172.21.15.37:0/953593744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32800550b0 msgr2=0x7f3280032a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.710+0000 7f326d7fa640 1 --2- 172.21.15.37:0/953593744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32800550b0 0x7f3280032a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3280022a30 tx=0x7f32780046c0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.710+0000 7f326d7fa640 1 -- 172.21.15.37:0/953593744 shutdown_connections 2024-09-17T04:32:54.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.710+0000 7f326d7fa640 1 --2- 172.21.15.37:0/953593744 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f32800df390 0x7f3280032f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.710+0000 7f326d7fa640 1 --2- 172.21.15.37:0/953593744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32800550b0 0x7f3280032a20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.710+0000 7f326d7fa640 1 --2- 172.21.15.37:0/953593744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3280004a70 0x7f32800d7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.710+0000 7f326d7fa640 1 -- 172.21.15.37:0/953593744 >> 172.21.15.37:0/953593744 conn(0x7f328001ddf0 msgr2=0x7f328004e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:54.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.710+0000 7f326d7fa640 1 -- 172.21.15.37:0/953593744 shutdown_connections 2024-09-17T04:32:54.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.710+0000 7f326d7fa640 1 -- 172.21.15.37:0/953593744 wait complete. 2024-09-17T04:32:54.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.712+0000 7ff9ba69b640 1 -- 172.21.15.37:0/1940827195 shutdown_connections 2024-09-17T04:32:54.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.713+0000 7ff9ba69b640 1 -- 172.21.15.37:0/1940827195 wait complete. 2024-09-17T04:32:54.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.719+0000 7faad6ffd640 1 -- 172.21.15.37:0/4036374517 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v8)=0 v8) ==== 117+0+0 (secure 0 0 0) 0x7faad8048df0 con 0x7faae015cc40 2024-09-17T04:32:54.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.721+0000 7faae850b640 1 -- 172.21.15.37:0/4036374517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faae015cc40 msgr2=0x7faae015d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.721+0000 7faae850b640 1 --2- 172.21.15.37:0/4036374517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faae015cc40 0x7faae015d050 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faad8009000 tx=0x7faad8004a40 comp rx=0 tx=0).stop 2024-09-17T04:32:54.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.721+0000 7faae850b640 1 -- 172.21.15.37:0/4036374517 shutdown_connections 2024-09-17T04:32:54.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.721+0000 7faae850b640 1 --2- 172.21.15.37:0/4036374517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faae015cc40 0x7faae015d050 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.721+0000 7faae850b640 1 --2- 172.21.15.37:0/4036374517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faae0154bb0 0x7faae0155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.721+0000 7faae850b640 1 --2- 172.21.15.37:0/4036374517 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7faae0073f70 0x7faae0201d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.721+0000 7faae850b640 1 -- 172.21.15.37:0/4036374517 >> 172.21.15.37:0/4036374517 conn(0x7faae00b20e0 msgr2=0x7faae0075f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:54.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.722+0000 7faae850b640 1 -- 172.21.15.37:0/4036374517 shutdown_connections 2024-09-17T04:32:54.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.722+0000 7faae850b640 1 -- 172.21.15.37:0/4036374517 wait complete. 2024-09-17T04:32:54.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.722+0000 7ff9ba69b640 1 Processor -- start 2024-09-17T04:32:54.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.722+0000 7ff9ba69b640 1 -- start start 2024-09-17T04:32:54.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.725+0000 7ff9ba69b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff9ac006070 0x7ff9ac0dc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.727+0000 7f87508b6640 1 -- 172.21.15.37:0/217492539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8744037490 msgr2=0x7f87440378a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.727+0000 7f87508b6640 1 --2- 172.21.15.37:0/217492539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8744037490 0x7f87440378a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f873c005fd0 tx=0x7f873c03b570 comp rx=0 tx=0).stop 2024-09-17T04:32:54.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.731+0000 7ff9ba69b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9ac051be0 0x7ff9ac0e0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.732+0000 7f87508b6640 1 -- 172.21.15.37:0/217492539 shutdown_connections 2024-09-17T04:32:54.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.732+0000 7f87508b6640 1 --2- 172.21.15.37:0/217492539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8744037490 0x7f87440378a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.732+0000 7f87508b6640 1 --2- 172.21.15.37:0/217492539 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f87440544d0 0x7f87440e3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.732+0000 7f87508b6640 1 -- 172.21.15.37:0/217492539 >> 172.21.15.37:0/217492539 conn(0x7f8744055e70 msgr2=0x7f874400e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:54.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.732+0000 7f87508b6640 1 -- 172.21.15.37:0/217492539 shutdown_connections 2024-09-17T04:32:54.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.734+0000 7ff9b3fff640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff9ac006070 0x7ff9ac0dc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.734+0000 7ff9b3fff640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff9ac006070 0x7ff9ac0dc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.37:40370/0 (socket says 172.21.15.37:40370) 2024-09-17T04:32:54.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.735+0000 7ff9b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9ac051be0 0x7ff9ac0e0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.735+0000 7ff9b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9ac051be0 0x7ff9ac0e0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56532/0 (socket says 172.21.15.37:56532) 2024-09-17T04:32:54.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.735+0000 7ff9b3fff640 1 -- 172.21.15.37:0/3058839357 learned_addr learned my addr 172.21.15.37:0/3058839357 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:32:54.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.735+0000 7ff9ba69b640 1 --2- 172.21.15.37:0/3058839357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9ac0e0bc0 0x7ff9ac0321e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.738+0000 7ff9ba69b640 1 -- 172.21.15.37:0/3058839357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac053100 con 0x7ff9ac051be0 2024-09-17T04:32:54.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.738+0000 7ff9b8c38640 1 --2- 172.21.15.37:0/3058839357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9ac0e0bc0 0x7ff9ac0321e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.738+0000 7ff9ba69b640 1 -- 172.21.15.37:0/3058839357 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac052f80 con 0x7ff9ac0e0bc0 2024-09-17T04:32:54.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.738+0000 7ff9ba69b640 1 -- 172.21.15.37:0/3058839357 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7ff9ac053280 con 0x7ff9ac006070 2024-09-17T04:32:54.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.745+0000 7ff9b37fe640 1 -- 172.21.15.37:0/3058839357 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff9ac006070 msgr2=0x7ff9ac0dc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.745+0000 7ff9b37fe640 1 --2- 172.21.15.37:0/3058839357 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff9ac006070 0x7ff9ac0dc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.745+0000 7ff9b37fe640 1 -- 172.21.15.37:0/3058839357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9ac0e0bc0 msgr2=0x7ff9ac0321e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.745+0000 7ff9b37fe640 1 --2- 172.21.15.37:0/3058839357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9ac0e0bc0 0x7ff9ac0321e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.745+0000 7ff9b37fe640 1 -- 172.21.15.37:0/3058839357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9ac04a710 con 0x7ff9ac051be0 2024-09-17T04:32:54.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.746+0000 7ff9b37fe640 1 --2- 172.21.15.37:0/3058839357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9ac051be0 0x7ff9ac0e0680 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000b780 tx=0x7ff9a000bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:54.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.746+0000 7ff9b8c38640 1 --2- 172.21.15.37:0/3058839357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9ac0e0bc0 0x7ff9ac0321e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:32:54.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.748+0000 7ff9b0ff9640 1 -- 172.21.15.37:0/3058839357 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0019070 con 0x7ff9ac051be0 2024-09-17T04:32:54.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.748+0000 7ff9b0ff9640 1 -- 172.21.15.37:0/3058839357 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9a000c810 con 0x7ff9ac051be0 2024-09-17T04:32:54.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.749+0000 7ff9b0ff9640 1 -- 172.21.15.37:0/3058839357 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a000cc60 con 0x7ff9ac051be0 2024-09-17T04:32:54.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.749+0000 7ff9ba69b640 1 -- 172.21.15.37:0/3058839357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9ac04a9f0 con 0x7ff9ac051be0 2024-09-17T04:32:54.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.749+0000 7ff9ba69b640 1 -- 172.21.15.37:0/3058839357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9ac0dcd40 con 0x7ff9ac051be0 2024-09-17T04:32:54.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.733+0000 7f87508b6640 1 -- 172.21.15.37:0/217492539 wait complete. 2024-09-17T04:32:54.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.751+0000 7f87508b6640 1 Processor -- start 2024-09-17T04:32:54.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.751+0000 7f87508b6640 1 -- start start 2024-09-17T04:32:54.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.754+0000 7ff9ba69b640 1 -- 172.21.15.37:0/3058839357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9ac03a600 con 0x7ff9ac051be0 2024-09-17T04:32:54.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.756+0000 7ff9b0ff9640 1 -- 172.21.15.37:0/3058839357 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff9a001c070 con 0x7ff9ac051be0 2024-09-17T04:32:54.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.756+0000 7ff9b0ff9640 1 -- 172.21.15.37:0/3058839357 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff9a0014780 con 0x7ff9ac051be0 2024-09-17T04:32:54.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.758+0000 7f4d6fb13640 1 -- 172.21.15.37:0/2289053516 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4d68073570 msgr2=0x7f4d680b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.758+0000 7f4d6fb13640 1 --2- 172.21.15.37:0/2289053516 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4d68073570 0x7f4d680b31a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4d60005fd0 tx=0x7f4d6002a930 comp rx=0 tx=0).stop 2024-09-17T04:32:54.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.759+0000 7f4d6fb13640 1 -- 172.21.15.37:0/2289053516 shutdown_connections 2024-09-17T04:32:54.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.759+0000 7f4d6fb13640 1 --2- 172.21.15.37:0/2289053516 >> v2:172.21.15.37:3300/0 conn(0x7f4d6816b210 0x7f4d6816b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.759+0000 7f4d6fb13640 1 --2- 172.21.15.37:0/2289053516 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4d68073570 0x7f4d680b31a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.759+0000 7f4d6fb13640 1 -- 172.21.15.37:0/2289053516 >> 172.21.15.37:0/2289053516 conn(0x7f4d6806d390 msgr2=0x7f4d680b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:54.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.759+0000 7f4d6fb13640 1 -- 172.21.15.37:0/2289053516 shutdown_connections 2024-09-17T04:32:54.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.761+0000 7ff9b0ff9640 1 -- 172.21.15.37:0/3058839357 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7ff9a0023030 con 0x7ff9ac051be0 2024-09-17T04:32:54.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.759+0000 7f4d6fb13640 1 -- 172.21.15.37:0/2289053516 wait complete. 2024-09-17T04:32:54.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.754+0000 7f87508b6640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8744037490 0x7f874404c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.766+0000 7f87508b6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87440538c0 0x7f874404b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.766+0000 7f87508b6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87440544d0 0x7f874404bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.766+0000 7f87508b6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8744005ff0 con 0x7f87440544d0 2024-09-17T04:32:54.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.766+0000 7f87508b6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8744038240 con 0x7f87440538c0 2024-09-17T04:32:54.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.766+0000 7f87508b6640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f8744006170 con 0x7f8744037490 2024-09-17T04:32:54.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.766+0000 7f874ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87440544d0 0x7f874404bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.767+0000 7f874ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87440544d0 0x7f874404bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56548/0 (socket says 172.21.15.37:56548) 2024-09-17T04:32:54.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.767+0000 7f874ad9d640 1 -- 172.21.15.37:0/3065825660 learned_addr learned my addr 172.21.15.37:0/3065825660 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:32:54.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.767+0000 7f874ad9d640 1 -- 172.21.15.37:0/3065825660 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8744037490 msgr2=0x7f874404c630 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T04:32:54.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.767+0000 7f874ad9d640 1 --2- 172.21.15.37:0/3065825660 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8744037490 0x7f874404c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.767+0000 7f874ad9d640 1 -- 172.21.15.37:0/3065825660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87440538c0 msgr2=0x7f874404b690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T04:32:54.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.767+0000 7f874ad9d640 1 --2- 172.21.15.37:0/3065825660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87440538c0 0x7f874404b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.767+0000 7f874ad9d640 1 -- 172.21.15.37:0/3065825660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f874404f050 con 0x7f87440544d0 2024-09-17T04:32:54.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.762+0000 7f4d6fb13640 1 Processor -- start 2024-09-17T04:32:54.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.768+0000 7f874ad9d640 1 --2- 172.21.15.37:0/3065825660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87440544d0 0x7f874404bbd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f873c044040 tx=0x7f873c004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:54.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.769+0000 7f873affd640 1 -- 172.21.15.37:0/3065825660 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c052070 con 0x7f87440544d0 2024-09-17T04:32:54.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.769+0000 7f873affd640 1 -- 172.21.15.37:0/3065825660 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f873c005310 con 0x7f87440544d0 2024-09-17T04:32:54.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.769+0000 7f87508b6640 1 -- 172.21.15.37:0/3065825660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f874404f330 con 0x7f87440544d0 2024-09-17T04:32:54.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.770+0000 7f87508b6640 1 -- 172.21.15.37:0/3065825660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8744039030 con 0x7f87440544d0 2024-09-17T04:32:54.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.770+0000 7fdbc2440640 1 -- 172.21.15.37:0/1501188426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdba8003a00 msgr2=0x7fdba8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.770+0000 7fdbc2440640 1 --2- 172.21.15.37:0/1501188426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdba8003a00 0x7fdba8003e70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdbac00c390 tx=0x7fdba8005830 comp rx=0 tx=0).stop 2024-09-17T04:32:54.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.770+0000 7fdbc1c3f640 1 -- 172.21.15.37:0/1501188426 reap_dead start 2024-09-17T04:32:54.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.771+0000 7fdbc2440640 1 -- 172.21.15.37:0/1501188426 shutdown_connections 2024-09-17T04:32:54.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.771+0000 7fdbc2440640 1 -- 172.21.15.37:0/1501188426 >> 172.21.15.37:0/1501188426 conn(0x7fdbb401e230 msgr2=0x7fdbb4039ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:54.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.771+0000 7f873affd640 1 -- 172.21.15.37:0/3065825660 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c0047a0 con 0x7f87440544d0 2024-09-17T04:32:54.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.771+0000 7f873affd640 1 -- 172.21.15.37:0/3065825660 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f873c004940 con 0x7f87440544d0 2024-09-17T04:32:54.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.771+0000 7f873affd640 1 -- 172.21.15.37:0/3065825660 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f873c04c630 con 0x7f87440544d0 2024-09-17T04:32:54.774 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi037') 2024-09-17T04:32:54.774 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi037') 2024-09-17T04:32:54.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.771+0000 7fdbc2440640 1 -- 172.21.15.37:0/1501188426 shutdown_connections 2024-09-17T04:32:54.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.777+0000 7f87508b6640 1 -- 172.21.15.37:0/3065825660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87440378a0 con 0x7f87440544d0 2024-09-17T04:32:54.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.768+0000 7f4d6fb13640 1 -- start start 2024-09-17T04:32:54.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.784+0000 7f4d6fb13640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d6806fd00 0x7f4d6815afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.781+0000 7fdbc2440640 1 -- 172.21.15.37:0/1501188426 wait complete. 2024-09-17T04:32:54.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.784+0000 7fdbc2440640 1 Processor -- start 2024-09-17T04:32:54.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.784+0000 7f4d6d8af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d6806fd00 0x7f4d6815afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.784+0000 7f4d6d8af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d6806fd00 0x7f4d6815afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56558/0 (socket says 172.21.15.37:56558) 2024-09-17T04:32:54.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.784+0000 7f4d6d8af640 1 -- 172.21.15.37:0/2506532410 learned_addr learned my addr 172.21.15.37:0/2506532410 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:32:54.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.784+0000 7f4d6fb13640 1 --2- 172.21.15.37:0/2506532410 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4d6816b210 0x7f4d6817c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.784+0000 7f4d6fb13640 1 --2- 172.21.15.37:0/2506532410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d6815b7e0 0x7f4d6817cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.785+0000 7f4d6fb13640 1 -- 172.21.15.37:0/2506532410 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6816bfa0 con 0x7f4d6806fd00 2024-09-17T04:32:54.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.785+0000 7f4d6fb13640 1 -- 172.21.15.37:0/2506532410 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6816be20 con 0x7f4d6815b7e0 2024-09-17T04:32:54.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.785+0000 7f4d6d0ae640 1 --2- 172.21.15.37:0/2506532410 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4d6816b210 0x7f4d6817c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.785+0000 7f4d6fb13640 1 -- 172.21.15.37:0/2506532410 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f4d6816c120 con 0x7f4d6816b210 2024-09-17T04:32:54.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.785+0000 7f4d6e0b0640 1 --2- 172.21.15.37:0/2506532410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d6815b7e0 0x7f4d6817cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.785+0000 7fdbc2440640 1 -- start start 2024-09-17T04:32:54.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.785+0000 7f4d6d8af640 1 -- 172.21.15.37:0/2506532410 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4d6816b210 msgr2=0x7f4d6817c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.785+0000 7f4d6d8af640 1 --2- 172.21.15.37:0/2506532410 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4d6816b210 0x7f4d6817c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.785+0000 7f4d6d8af640 1 -- 172.21.15.37:0/2506532410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d6815b7e0 msgr2=0x7f4d6817cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.785+0000 7f4d6d8af640 1 --2- 172.21.15.37:0/2506532410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d6815b7e0 0x7f4d6817cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.785+0000 7f4d6d8af640 1 -- 172.21.15.37:0/2506532410 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6820e9e0 con 0x7f4d6806fd00 2024-09-17T04:32:54.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.786+0000 7f4d6d8af640 1 --2- 172.21.15.37:0/2506532410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d6806fd00 0x7f4d6815afe0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4d60033040 tx=0x7f4d60003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:54.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.787+0000 7f4d5e7fc640 1 -- 172.21.15.37:0/2506532410 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d60004d40 con 0x7f4d6806fd00 2024-09-17T04:32:54.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.787+0000 7f4d5e7fc640 1 -- 172.21.15.37:0/2506532410 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d60036070 con 0x7f4d6806fd00 2024-09-17T04:32:54.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.787+0000 7f4d6fb13640 1 -- 172.21.15.37:0/2506532410 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d680cfa20 con 0x7f4d6806fd00 2024-09-17T04:32:54.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.787+0000 7fdbc2440640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdba8003a00 0x7fdbb4187220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.787+0000 7fdbc2440640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbb404b460 0x7fdbb404da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.787+0000 7fdbc2440640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbb404b870 0x7fdbb404df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.787+0000 7f4d6fb13640 1 -- 172.21.15.37:0/2506532410 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d6815d8f0 con 0x7f4d6806fd00 2024-09-17T04:32:54.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.788+0000 7fdbc0c3d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbb404b460 0x7fdbb404da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.788+0000 7fdbc0c3d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbb404b460 0x7fdbb404da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.37:58284/0 (socket says 172.21.15.37:58284) 2024-09-17T04:32:54.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.788+0000 7fdbc0c3d640 1 -- 172.21.15.37:0/1556303495 learned_addr learned my addr 172.21.15.37:0/1556303495 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:32:54.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.788+0000 7fdbc1c3f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbb404b870 0x7fdbb404df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.788+0000 7fdbc2440640 1 -- 172.21.15.37:0/1556303495 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb40dedf0 con 0x7fdbb404b870 2024-09-17T04:32:54.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.788+0000 7fdbc2440640 1 -- 172.21.15.37:0/1556303495 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb40dec70 con 0x7fdbb404b460 2024-09-17T04:32:54.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.788+0000 7fdbc2440640 1 -- 172.21.15.37:0/1556303495 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fdbb40def70 con 0x7fdba8003a00 2024-09-17T04:32:54.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.789+0000 7fdbc1c3f640 1 -- 172.21.15.37:0/1556303495 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdba8003a00 msgr2=0x7fdbb4187220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T04:32:54.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.789+0000 7fdbc1c3f640 1 --2- 172.21.15.37:0/1556303495 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdba8003a00 0x7fdbb4187220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.789+0000 7fdbc1c3f640 1 -- 172.21.15.37:0/1556303495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbb404b460 msgr2=0x7fdbb404da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.790+0000 7f4d5e7fc640 1 -- 172.21.15.37:0/2506532410 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d60004420 con 0x7f4d6806fd00 2024-09-17T04:32:54.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.789+0000 7fdbc1c3f640 1 --2- 172.21.15.37:0/1556303495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbb404b460 0x7fdbb404da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.790+0000 7f4d5e7fc640 1 -- 172.21.15.37:0/2506532410 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4d60030070 con 0x7f4d6806fd00 2024-09-17T04:32:54.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.789+0000 7fdbc1c3f640 1 -- 172.21.15.37:0/1556303495 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb4187760 con 0x7fdbb404b870 2024-09-17T04:32:54.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.790+0000 7f4d5e7fc640 1 -- 172.21.15.37:0/2506532410 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4d600444e0 con 0x7f4d6806fd00 2024-09-17T04:32:54.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.791+0000 7f873affd640 1 -- 172.21.15.37:0/3065825660 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f873c004c30 con 0x7f87440544d0 2024-09-17T04:32:54.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.790+0000 7fdbc1c3f640 1 --2- 172.21.15.37:0/1556303495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbb404b870 0x7fdbb404df60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4037b50 tx=0x7fdbb800ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:54.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.794+0000 7fdbb1ffb640 1 -- 172.21.15.37:0/1556303495 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8004020 con 0x7fdbb404b870 2024-09-17T04:32:54.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.795+0000 7fdbc2440640 1 -- 172.21.15.37:0/1556303495 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb4048e20 con 0x7fdbb404b870 2024-09-17T04:32:54.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.791+0000 7f4d53fff640 1 -- 172.21.15.37:0/2506532410 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d48005290 con 0x7f4d6806fd00 2024-09-17T04:32:54.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.794+0000 7fdbb1ffb640 1 -- 172.21.15.37:0/1556303495 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbb80041c0 con 0x7fdbb404b870 2024-09-17T04:32:54.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.796+0000 7fdbc2440640 1 -- 172.21.15.37:0/1556303495 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb40493e0 con 0x7fdbb404b870 2024-09-17T04:32:54.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.798+0000 7fa944e30640 1 -- 172.21.15.37:0/3570768943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa940073570 msgr2=0x7fa9400b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.798+0000 7fa944e30640 1 --2- 172.21.15.37:0/3570768943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa940073570 0x7fa9400b31a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa930009820 tx=0x7fa93002c730 comp rx=0 tx=0).stop 2024-09-17T04:32:54.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.798+0000 7fa944e30640 1 -- 172.21.15.37:0/3570768943 shutdown_connections 2024-09-17T04:32:54.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.798+0000 7fa944e30640 1 --2- 172.21.15.37:0/3570768943 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa94016b210 0x7fa94016b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.798+0000 7fa944e30640 1 --2- 172.21.15.37:0/3570768943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa940073570 0x7fa9400b31a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.798+0000 7fa944e30640 1 -- 172.21.15.37:0/3570768943 >> 172.21.15.37:0/3570768943 conn(0x7fa94006d390 msgr2=0x7fa9400b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:54.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.800+0000 7fa944e30640 1 -- 172.21.15.37:0/3570768943 shutdown_connections 2024-09-17T04:32:54.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.801+0000 7f4d5e7fc640 1 -- 172.21.15.37:0/2506532410 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f4d60030a20 con 0x7f4d6806fd00 2024-09-17T04:32:54.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.806+0000 7fdbb1ffb640 1 -- 172.21.15.37:0/1556303495 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8007770 con 0x7fdbb404b870 2024-09-17T04:32:54.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.808+0000 7fdbb1ffb640 1 -- 172.21.15.37:0/1556303495 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdbb8019020 con 0x7fdbb404b870 2024-09-17T04:32:54.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.809+0000 7fdbb1ffb640 1 -- 172.21.15.37:0/1556303495 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdbb800f570 con 0x7fdbb404b870 2024-09-17T04:32:54.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.800+0000 7fa944e30640 1 -- 172.21.15.37:0/3570768943 wait complete. 2024-09-17T04:32:54.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.804+0000 7fa944e30640 1 Processor -- start 2024-09-17T04:32:54.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.804+0000 7fa944e30640 1 -- start start 2024-09-17T04:32:54.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.808+0000 7fa944e30640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa94006fd00 0x7fa940168100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.813+0000 7fdba77fe640 1 -- 172.21.15.37:0/1556303495 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb404bc80 con 0x7fdbb404b870 2024-09-17T04:32:54.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.816+0000 7fd38e9a8640 1 -- 172.21.15.37:0/1605724816 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd370003ae0 msgr2=0x7fd370003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.816+0000 7fd38e9a8640 1 --2- 172.21.15.37:0/1605724816 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd370003ae0 0x7fd370003f10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd380004740 tx=0x7fd370005830 comp rx=0 tx=0).stop 2024-09-17T04:32:54.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.816+0000 7fd38cf45640 1 -- 172.21.15.37:0/1605724816 reap_dead start 2024-09-17T04:32:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.808+0000 7fa944e30640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa94016b210 0x7fa940169ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.808+0000 7fa944e30640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa94015bf90 0x7fa94015c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.808+0000 7fa944e30640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa94016bf70 con 0x7fa94006fd00 2024-09-17T04:32:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.808+0000 7fa944e30640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa94016bdf0 con 0x7fa94015bf90 2024-09-17T04:32:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.808+0000 7fa944e30640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fa94016c0f0 con 0x7fa94016b210 2024-09-17T04:32:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.808+0000 7fa93e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa94006fd00 0x7fa940168100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.809+0000 7fa93e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa94006fd00 0x7fa940168100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56592/0 (socket says 172.21.15.37:56592) 2024-09-17T04:32:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.809+0000 7fa93e59c640 1 -- 172.21.15.37:0/2293969086 learned_addr learned my addr 172.21.15.37:0/2293969086 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:32:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.809+0000 7fa93e59c640 1 -- 172.21.15.37:0/2293969086 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa94016b210 msgr2=0x7fa940169ec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T04:32:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.809+0000 7fa93e59c640 1 --2- 172.21.15.37:0/2293969086 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa94016b210 0x7fa940169ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.809+0000 7fa93e59c640 1 -- 172.21.15.37:0/2293969086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa94015bf90 msgr2=0x7fa94015c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.809+0000 7fa93e59c640 1 --2- 172.21.15.37:0/2293969086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa94015bf90 0x7fa94015c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.809+0000 7fa93e59c640 1 -- 172.21.15.37:0/2293969086 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94015faa0 con 0x7fa94006fd00 2024-09-17T04:32:54.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.809+0000 7fa93e59c640 1 --2- 172.21.15.37:0/2293969086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa94006fd00 0x7fa940168100 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa930009770 tx=0x7fa93000e390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:54.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.810+0000 7fa92effd640 1 -- 172.21.15.37:0/2293969086 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa930041cc0 con 0x7fa94006fd00 2024-09-17T04:32:54.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.810+0000 7fa92effd640 1 -- 172.21.15.37:0/2293969086 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa930004020 con 0x7fa94006fd00 2024-09-17T04:32:54.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.810+0000 7fa92effd640 1 -- 172.21.15.37:0/2293969086 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa930004450 con 0x7fa94006fd00 2024-09-17T04:32:54.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.810+0000 7fa944e30640 1 -- 172.21.15.37:0/2293969086 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94015ece0 con 0x7fa94006fd00 2024-09-17T04:32:54.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.810+0000 7fa944e30640 1 -- 172.21.15.37:0/2293969086 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa94015f2a0 con 0x7fa94006fd00 2024-09-17T04:32:54.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.818+0000 7f6dcdfc3640 1 -- 172.21.15.37:0/3203204881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dc8070410 msgr2=0x7f6dc816b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.818+0000 7f6dc67fc640 1 -- 172.21.15.37:0/3203204881 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc034680 con 0x7f6dc8070410 2024-09-17T04:32:54.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.818+0000 7f6dcdfc3640 1 --2- 172.21.15.37:0/3203204881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dc8070410 0x7f6dc816b300 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8070900 tx=0x7f6dbc032b30 comp rx=0 tx=0).stop 2024-09-17T04:32:54.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.818+0000 7f6dcdfc3640 1 -- 172.21.15.37:0/3203204881 shutdown_connections 2024-09-17T04:32:54.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.818+0000 7f6dcdfc3640 1 --2- 172.21.15.37:0/3203204881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dc8070410 0x7f6dc816b300 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.818+0000 7f6dcdfc3640 1 --2- 172.21.15.37:0/3203204881 >> v2:172.21.15.92:3300/0 conn(0x7f6dc816c3e0 0x7f6dc8073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.818+0000 7f6dcdfc3640 1 -- 172.21.15.37:0/3203204881 >> 172.21.15.37:0/3203204881 conn(0x7f6dc806d390 msgr2=0x7f6dc80b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:54.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.819+0000 7f6dcdfc3640 1 -- 172.21.15.37:0/3203204881 shutdown_connections 2024-09-17T04:32:54.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.819+0000 7f6dcdfc3640 1 -- 172.21.15.37:0/3203204881 wait complete. 2024-09-17T04:32:54.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.822+0000 7f6dcdfc3640 1 Processor -- start 2024-09-17T04:32:54.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.824+0000 7fd38e9a8640 1 -- 172.21.15.37:0/1605724816 shutdown_connections 2024-09-17T04:32:54.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.824+0000 7fd38e9a8640 1 -- 172.21.15.37:0/1605724816 >> 172.21.15.37:0/1605724816 conn(0x7fd38809d800 msgr2=0x7fd3880d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:54.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.825+0000 7fd38e9a8640 1 -- 172.21.15.37:0/1605724816 shutdown_connections 2024-09-17T04:32:54.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.825+0000 7fd38e9a8640 1 -- 172.21.15.37:0/1605724816 wait complete. 2024-09-17T04:32:54.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.820+0000 7fdbb1ffb640 1 -- 172.21.15.37:0/1556303495 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fdbb4187760 con 0x7fdbb404b870 2024-09-17T04:32:54.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.824+0000 7fa92effd640 1 -- 172.21.15.37:0/2293969086 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa930033910 con 0x7fa94006fd00 2024-09-17T04:32:54.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.824+0000 7fa92effd640 1 -- 172.21.15.37:0/2293969086 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa9300416c0 con 0x7fa94006fd00 2024-09-17T04:32:54.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.828+0000 7fd38e9a8640 1 Processor -- start 2024-09-17T04:32:54.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.828+0000 7fd38e9a8640 1 -- start start 2024-09-17T04:32:54.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.829+0000 7f6dcdfc3640 1 -- start start 2024-09-17T04:32:54.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.831+0000 7f6dcdfc3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dc8070410 0x7f6dc81633c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.831+0000 7f6dcdfc3640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6dc80b31a0 0x7f6dc80ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.832+0000 7f6dcdfc3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dc816c3e0 0x7f6dc80cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.832+0000 7f6dc77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dc8070410 0x7f6dc81633c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.832+0000 7f6dc6ffd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6dc80b31a0 0x7f6dc80ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.832+0000 7f6dc6ffd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6dc80b31a0 0x7f6dc80ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.37:40470/0 (socket says 172.21.15.37:40470) 2024-09-17T04:32:54.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.832+0000 7f6dc6ffd640 1 -- 172.21.15.37:0/3830474725 learned_addr learned my addr 172.21.15.37:0/3830474725 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:32:54.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.832+0000 7f6dcdfc3640 1 -- 172.21.15.37:0/3830474725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc816bbd0 con 0x7f6dc816c3e0 2024-09-17T04:32:54.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.832+0000 7f6dcdfc3640 1 -- 172.21.15.37:0/3830474725 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc816ba50 con 0x7f6dc8070410 2024-09-17T04:32:54.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.832+0000 7f6dcdfc3640 1 -- 172.21.15.37:0/3830474725 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f6dc816bd50 con 0x7f6dc80b31a0 2024-09-17T04:32:54.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.833+0000 7f6dc6ffd640 1 -- 172.21.15.37:0/3830474725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dc8070410 msgr2=0x7f6dc81633c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.833+0000 7f6dc6ffd640 1 --2- 172.21.15.37:0/3830474725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dc8070410 0x7f6dc81633c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.833+0000 7f6dc6ffd640 1 -- 172.21.15.37:0/3830474725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dc816c3e0 msgr2=0x7f6dc80cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.833+0000 7f6dc6ffd640 1 --2- 172.21.15.37:0/3830474725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dc816c3e0 0x7f6dc80cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.833+0000 7f6dc6ffd640 1 -- 172.21.15.37:0/3830474725 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc8163f00 con 0x7f6dc80b31a0 2024-09-17T04:32:54.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.835+0000 7fa944e30640 1 -- 172.21.15.37:0/2293969086 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa924005290 con 0x7fa94006fd00 2024-09-17T04:32:54.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.836+0000 7fd38e9a8640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd370003ae0 0x7fd3881e6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.836+0000 7fd38e9a8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3881e7720 0x7fd3880b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.836+0000 7fd387fff640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd370003ae0 0x7fd3881e6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.836+0000 7fd387fff640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd370003ae0 0x7fd3881e6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.37:40480/0 (socket says 172.21.15.37:40480) 2024-09-17T04:32:54.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.836+0000 7fd387fff640 1 -- 172.21.15.37:0/2993994265 learned_addr learned my addr 172.21.15.37:0/2993994265 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:32:54.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.836+0000 7fd38e9a8640 1 --2- 172.21.15.37:0/2993994265 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3880b20c0 0x7fd3881f82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.837+0000 7fd38e9a8640 1 -- 172.21.15.37:0/2993994265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd3880b3a20 con 0x7fd3881e7720 2024-09-17T04:32:54.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.837+0000 7fd38cf45640 1 --2- 172.21.15.37:0/2993994265 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3880b20c0 0x7fd3881f82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.837+0000 7fd38e9a8640 1 -- 172.21.15.37:0/2993994265 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd3880b5d20 con 0x7fd3880b20c0 2024-09-17T04:32:54.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.837+0000 7fd38e9a8640 1 -- 172.21.15.37:0/2993994265 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fd3880b3d40 con 0x7fd370003ae0 2024-09-17T04:32:54.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.837+0000 7fd387fff640 1 -- 172.21.15.37:0/2993994265 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3880b20c0 msgr2=0x7fd3881f82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.837+0000 7fd387fff640 1 --2- 172.21.15.37:0/2993994265 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3880b20c0 0x7fd3881f82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.837+0000 7fd387fff640 1 -- 172.21.15.37:0/2993994265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3881e7720 msgr2=0x7fd3880b1b80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T04:32:54.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.837+0000 7fd387fff640 1 --2- 172.21.15.37:0/2993994265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3881e7720 0x7fd3880b1b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.837+0000 7fd387fff640 1 -- 172.21.15.37:0/2993994265 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3880b24d0 con 0x7fd370003ae0 2024-09-17T04:32:54.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.838+0000 7fd387fff640 1 --2- 172.21.15.37:0/2993994265 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd370003ae0 0x7fd3881e6df0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd380000f80 tx=0x7fd380001aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:54.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.841+0000 7f6dc6ffd640 1 --2- 172.21.15.37:0/3830474725 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6dc80b31a0 0x7f6dc80ceaa0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000cbc0 tx=0x7f6dc000a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:54.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.841+0000 7f6daffff640 1 -- 172.21.15.37:0/3830474725 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0016020 con 0x7f6dc80b31a0 2024-09-17T04:32:54.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.841+0000 7f6daffff640 1 -- 172.21.15.37:0/3830474725 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6dc000ae30 con 0x7f6dc80b31a0 2024-09-17T04:32:54.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.841+0000 7f6dcdfc3640 1 -- 172.21.15.37:0/3830474725 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc80cdce0 con 0x7f6dc80b31a0 2024-09-17T04:32:54.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.841+0000 7f6dcdfc3640 1 -- 172.21.15.37:0/3830474725 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc80ce2a0 con 0x7f6dc80b31a0 2024-09-17T04:32:54.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.839+0000 7fa92effd640 1 -- 172.21.15.37:0/2293969086 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fa9300418c0 con 0x7fa94006fd00 2024-09-17T04:32:54.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.843+0000 7f6daffff640 1 -- 172.21.15.37:0/3830474725 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0012680 con 0x7f6dc80b31a0 2024-09-17T04:32:54.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.844+0000 7f6daffff640 1 -- 172.21.15.37:0/3830474725 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6dc0019020 con 0x7f6dc80b31a0 2024-09-17T04:32:54.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.844+0000 7f6daffff640 1 -- 172.21.15.37:0/3830474725 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6dc001a3b0 con 0x7f6dc80b31a0 2024-09-17T04:32:54.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.847+0000 7fd384ff9640 1 -- 172.21.15.37:0/2993994265 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380001cd0 con 0x7fd370003ae0 2024-09-17T04:32:54.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.847+0000 7fd384ff9640 1 -- 172.21.15.37:0/2993994265 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd380001e70 con 0x7fd370003ae0 2024-09-17T04:32:54.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.847+0000 7fd38e9a8640 1 -- 172.21.15.37:0/2993994265 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3881e7f50 con 0x7fd370003ae0 2024-09-17T04:32:54.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.847+0000 7fd38e9a8640 1 -- 172.21.15.37:0/2993994265 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3880a0600 con 0x7fd370003ae0 2024-09-17T04:32:54.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.848+0000 7fce377fe640 1 -- 172.21.15.37:0/2837040995 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7fce4401d110 con 0x7fce44178e70 2024-09-17T04:32:54.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.848+0000 7fd384ff9640 1 -- 172.21.15.37:0/2993994265 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380007270 con 0x7fd370003ae0 2024-09-17T04:32:54.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.848+0000 7f6dadffb640 1 -- 172.21.15.37:0/3830474725 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6da8005290 con 0x7f6dc80b31a0 2024-09-17T04:32:54.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.849+0000 7fd384ff9640 1 -- 172.21.15.37:0/2993994265 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd380032040 con 0x7fd370003ae0 2024-09-17T04:32:54.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.849+0000 7fd384ff9640 1 -- 172.21.15.37:0/2993994265 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd3800408a0 con 0x7fd370003ae0 2024-09-17T04:32:54.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.850+0000 7fd36e7fc640 1 -- 172.21.15.37:0/2993994265 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd368005290 con 0x7fd370003ae0 2024-09-17T04:32:54.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.853+0000 7fce41ffb640 1 -- 172.21.15.37:0/2837040995 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v9)=0 v9) ==== 106+0+0 (secure 0 0 0) 0x7fce3c040dc0 con 0x7fce44178e70 2024-09-17T04:32:54.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.854+0000 7f8cad7fa640 1 -- 172.21.15.37:0/4077293742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f8ca8005850 con 0x7f8cc00e1640 2024-09-17T04:32:54.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.855+0000 7fd384ff9640 1 -- 172.21.15.37:0/2993994265 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd3880b24d0 con 0x7fd370003ae0 2024-09-17T04:32:54.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.857+0000 7fb044d7b640 1 -- 172.21.15.37:0/1201566856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0400b3790 msgr2=0x7fb0400b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.857+0000 7fb044d7b640 1 --2- 172.21.15.37:0/1201566856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0400b3790 0x7fb0400b3ba0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb04006dd10 tx=0x7fb03402a4b0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.861+0000 7fb044d7b640 1 -- 172.21.15.37:0/1201566856 shutdown_connections 2024-09-17T04:32:54.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.861+0000 7fb044d7b640 1 --2- 172.21.15.37:0/1201566856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0400b3790 0x7fb0400b3ba0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.861+0000 7fb044d7b640 1 --2- 172.21.15.37:0/1201566856 >> v2:172.21.15.92:3300/0 conn(0x7fb0400745e0 0x7fb0400749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.861+0000 7fb044d7b640 1 -- 172.21.15.37:0/1201566856 >> 172.21.15.37:0/1201566856 conn(0x7fb04006d390 msgr2=0x7fb0400b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:54.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.864+0000 7fce53569640 1 -- 172.21.15.37:0/2837040995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce44178e70 msgr2=0x7fce4401e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.864+0000 7fce53569640 1 --2- 172.21.15.37:0/2837040995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce44178e70 0x7fce4401e440 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009920 tx=0x7fce3c033ef0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.864+0000 7fce53569640 1 -- 172.21.15.37:0/2837040995 shutdown_connections 2024-09-17T04:32:54.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.864+0000 7fce53569640 1 --2- 172.21.15.37:0/2837040995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce44178e70 0x7fce4401e440 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.864+0000 7fce53569640 1 --2- 172.21.15.37:0/2837040995 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce440df1b0 0x7fce44178930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.864+0000 7fce53569640 1 --2- 172.21.15.37:0/2837040995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce44009bd0 0x7fce440dd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.864+0000 7fce53569640 1 -- 172.21.15.37:0/2837040995 >> 172.21.15.37:0/2837040995 conn(0x7fce44051450 msgr2=0x7fce44034500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:54.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.865+0000 7fb044d7b640 1 -- 172.21.15.37:0/1201566856 shutdown_connections 2024-09-17T04:32:54.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.865+0000 7fce53569640 1 -- 172.21.15.37:0/2837040995 shutdown_connections 2024-09-17T04:32:54.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.865+0000 7fce53569640 1 -- 172.21.15.37:0/2837040995 wait complete. 2024-09-17T04:32:54.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.867+0000 7f8caf7fe640 1 -- 172.21.15.37:0/4077293742 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v10)=0 v10) ==== 115+0+0 (secure 0 0 0) 0x7f8cc0006ab0 con 0x7f8cc00e1640 2024-09-17T04:32:54.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.871+0000 7fb044d7b640 1 -- 172.21.15.37:0/1201566856 wait complete. 2024-09-17T04:32:54.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.873+0000 7f8ccd539640 1 -- 172.21.15.37:0/4077293742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cc00e1640 msgr2=0x7f8cc00e1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.873+0000 7f8ccd539640 1 --2- 172.21.15.37:0/4077293742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cc00e1640 0x7f8cc00e1a50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8004170 tx=0x7f8cb8003680 comp rx=0 tx=0).stop 2024-09-17T04:32:54.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.874+0000 7fb044d7b640 1 Processor -- start 2024-09-17T04:32:54.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.875+0000 7f8ccd539640 1 -- 172.21.15.37:0/4077293742 shutdown_connections 2024-09-17T04:32:54.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.875+0000 7f8ccd539640 1 --2- 172.21.15.37:0/4077293742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cc00d7f00 0x7f8cc00d8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.875+0000 7f8ccd539640 1 --2- 172.21.15.37:0/4077293742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cc00e1640 0x7f8cc00e1a50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.875+0000 7f8ccd539640 1 --2- 172.21.15.37:0/4077293742 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8cb0004800 0x7f8cc00320f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.875+0000 7f8ccd539640 1 -- 172.21.15.37:0/4077293742 >> 172.21.15.37:0/4077293742 conn(0x7f8cc001d9d0 msgr2=0x7f8cc0053110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:54.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.876+0000 7f8ccd539640 1 -- 172.21.15.37:0/4077293742 shutdown_connections 2024-09-17T04:32:54.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.876+0000 7f8ccd539640 1 -- 172.21.15.37:0/4077293742 wait complete. 2024-09-17T04:32:54.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.875+0000 7fb044d7b640 1 -- start start 2024-09-17T04:32:54.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.860+0000 7f6daffff640 1 -- 172.21.15.37:0/3830474725 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f6dc0020020 con 0x7f6dc80b31a0 2024-09-17T04:32:54.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.877+0000 7fb044d7b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0400745e0 0x7fb0401fa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.877+0000 7fb044d7b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0400b3790 0x7fb0400acbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.877+0000 7fb044d7b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0401621b0 0x7fb040153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.878+0000 7fb03dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0400b3790 0x7fb0400acbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.878+0000 7fb03dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0400b3790 0x7fb0400acbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56646/0 (socket says 172.21.15.37:56646) 2024-09-17T04:32:54.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.878+0000 7fb03e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0400745e0 0x7fb0401fa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.878+0000 7fb044d7b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb040162e50 con 0x7fb0400b3790 2024-09-17T04:32:54.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.878+0000 7fb044d7b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb040162cd0 con 0x7fb0400745e0 2024-09-17T04:32:54.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.878+0000 7fb044d7b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fb040162fd0 con 0x7fb0401621b0 2024-09-17T04:32:54.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.879+0000 7fb03ed9d640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0401621b0 0x7fb040153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.879+0000 7fb03ed9d640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0401621b0 0x7fb040153ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.37:40482/0 (socket says 172.21.15.37:40482) 2024-09-17T04:32:54.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.879+0000 7fb03ed9d640 1 -- 172.21.15.37:0/3104482230 learned_addr learned my addr 172.21.15.37:0/3104482230 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:32:54.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.879+0000 7fb03e59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0400745e0 0x7fb0401fa3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.37:58342/0 (socket says 172.21.15.37:58342) 2024-09-17T04:32:54.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.881+0000 7fb03ed9d640 1 -- 172.21.15.37:0/3104482230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0400745e0 msgr2=0x7fb0401fa3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.881+0000 7fb03ed9d640 1 --2- 172.21.15.37:0/3104482230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0400745e0 0x7fb0401fa3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.881+0000 7fb03ed9d640 1 -- 172.21.15.37:0/3104482230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0400b3790 msgr2=0x7fb0400acbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.881+0000 7fb03ed9d640 1 --2- 172.21.15.37:0/3104482230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0400b3790 0x7fb0400acbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.881+0000 7fb03ed9d640 1 -- 172.21.15.37:0/3104482230 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0401565d0 con 0x7fb0401621b0 2024-09-17T04:32:54.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.880+0000 7f09dde0e640 1 -- 172.21.15.37:0/2104789277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09d80b3aa0 msgr2=0x7f09d80b4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.880+0000 7f09dde0e640 1 --2- 172.21.15.37:0/2104789277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09d80b3aa0 0x7f09d80b4960 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f09d80d0e90 tx=0x7f09c402a440 comp rx=0 tx=0).stop 2024-09-17T04:32:54.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.881+0000 7f09dde0e640 1 -- 172.21.15.37:0/2104789277 shutdown_connections 2024-09-17T04:32:54.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.881+0000 7f09dde0e640 1 --2- 172.21.15.37:0/2104789277 >> v2:172.21.15.37:3300/0 conn(0x7f09d815e860 0x7f09d81510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.881+0000 7f09dde0e640 1 --2- 172.21.15.37:0/2104789277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09d80b3aa0 0x7f09d80b4960 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.881+0000 7f09dde0e640 1 -- 172.21.15.37:0/2104789277 >> 172.21.15.37:0/2104789277 conn(0x7f09d80b1c10 msgr2=0x7f09d80b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:54.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.881+0000 7f09dde0e640 1 -- 172.21.15.37:0/2104789277 shutdown_connections 2024-09-17T04:32:54.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.882+0000 7fb03ed9d640 1 --2- 172.21.15.37:0/3104482230 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0401621b0 0x7fb040153ae0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb03800b9c0 tx=0x7fb03800be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:54.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.883+0000 7fb02effd640 1 -- 172.21.15.37:0/3104482230 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038019070 con 0x7fb0401621b0 2024-09-17T04:32:54.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.883+0000 7fb044d7b640 1 -- 172.21.15.37:0/3104482230 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb040156b60 con 0x7fb0401621b0 2024-09-17T04:32:54.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.883+0000 7fb044d7b640 1 -- 172.21.15.37:0/3104482230 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb040157120 con 0x7fb0401621b0 2024-09-17T04:32:54.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.883+0000 7fb02effd640 1 -- 172.21.15.37:0/3104482230 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb03800ccb0 con 0x7fb0401621b0 2024-09-17T04:32:54.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.883+0000 7fb02effd640 1 -- 172.21.15.37:0/3104482230 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0380146a0 con 0x7fb0401621b0 2024-09-17T04:32:54.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.883+0000 7fb02effd640 1 -- 172.21.15.37:0/3104482230 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb03801c070 con 0x7fb0401621b0 2024-09-17T04:32:54.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.885+0000 7fb02effd640 1 -- 172.21.15.37:0/3104482230 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb03801d3b0 con 0x7fb0401621b0 2024-09-17T04:32:54.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.888+0000 7fb044d7b640 1 -- 172.21.15.37:0/3104482230 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb024005290 con 0x7fb0401621b0 2024-09-17T04:32:54.890 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi037') 2024-09-17T04:32:54.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.882+0000 7f09dde0e640 1 -- 172.21.15.37:0/2104789277 wait complete. 2024-09-17T04:32:54.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.885+0000 7f09dde0e640 1 Processor -- start 2024-09-17T04:32:54.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.887+0000 7f09dde0e640 1 -- start start 2024-09-17T04:32:54.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.889+0000 7f09dde0e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09d8070c80 0x7f09d8142360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.889+0000 7f09dde0e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f09d80b3aa0 0x7f09d815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.889+0000 7f09dde0e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09d815da00 0x7f09d8154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.889+0000 7f09d77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09d8070c80 0x7f09d8142360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.890+0000 7f09d77fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09d8070c80 0x7f09d8142360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.37:58358/0 (socket says 172.21.15.37:58358) 2024-09-17T04:32:54.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.890+0000 7f09d6ffd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f09d80b3aa0 0x7f09d815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.890+0000 7f09d6ffd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f09d80b3aa0 0x7f09d815d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.37:40492/0 (socket says 172.21.15.37:40492) 2024-09-17T04:32:54.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.890+0000 7f09d6ffd640 1 -- 172.21.15.37:0/572527964 learned_addr learned my addr 172.21.15.37:0/572527964 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:32:54.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.890+0000 7f09dde0e640 1 -- 172.21.15.37:0/572527964 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f09d806eeb0 con 0x7f09d815da00 2024-09-17T04:32:54.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.890+0000 7f09dde0e640 1 -- 172.21.15.37:0/572527964 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f09d806ed30 con 0x7f09d8070c80 2024-09-17T04:32:54.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.890+0000 7f09dde0e640 1 -- 172.21.15.37:0/572527964 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f09d806f030 con 0x7f09d80b3aa0 2024-09-17T04:32:54.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.891+0000 7fb02effd640 1 -- 172.21.15.37:0/3104482230 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fb038023080 con 0x7fb0401621b0 2024-09-17T04:32:54.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.892+0000 7f09d7fff640 1 --2- 172.21.15.37:0/572527964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09d815da00 0x7f09d8154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.892+0000 7f09d6ffd640 1 -- 172.21.15.37:0/572527964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09d8070c80 msgr2=0x7f09d8142360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.892+0000 7f09d6ffd640 1 --2- 172.21.15.37:0/572527964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09d8070c80 0x7f09d8142360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.892+0000 7f09d6ffd640 1 -- 172.21.15.37:0/572527964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09d815da00 msgr2=0x7f09d8154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.892+0000 7f09d6ffd640 1 --2- 172.21.15.37:0/572527964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09d815da00 0x7f09d8154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.892+0000 7f09d6ffd640 1 -- 172.21.15.37:0/572527964 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d8155b20 con 0x7f09d80b3aa0 2024-09-17T04:32:54.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.895+0000 7f09d6ffd640 1 --2- 172.21.15.37:0/572527964 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f09d80b3aa0 0x7f09d815d4c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00c730 tx=0x7f09cc00cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:54.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.896+0000 7f09c3fff640 1 -- 172.21.15.37:0/572527964 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc017070 con 0x7f09d80b3aa0 2024-09-17T04:32:54.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.896+0000 7f09c3fff640 1 -- 172.21.15.37:0/572527964 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09cc005380 con 0x7f09d80b3aa0 2024-09-17T04:32:54.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.896+0000 7f09c3fff640 1 -- 172.21.15.37:0/572527964 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc003850 con 0x7f09d80b3aa0 2024-09-17T04:32:54.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.900+0000 7f09dde0e640 1 -- 172.21.15.37:0/572527964 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d8155dc0 con 0x7f09d80b3aa0 2024-09-17T04:32:54.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.900+0000 7f09dde0e640 1 -- 172.21.15.37:0/572527964 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d8156380 con 0x7f09d80b3aa0 2024-09-17T04:32:54.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.900+0000 7f09c3fff640 1 -- 172.21.15.37:0/572527964 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f09cc01a070 con 0x7f09d80b3aa0 2024-09-17T04:32:54.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.900+0000 7f09c3fff640 1 -- 172.21.15.37:0/572527964 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f09cc01c780 con 0x7f09d80b3aa0 2024-09-17T04:32:54.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.899+0000 7f0f8bc77640 1 -- 172.21.15.37:0/519721077 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0f8406fbe0 msgr2=0x7f0f8406fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.899+0000 7f0f8bc77640 1 --2- 172.21.15.37:0/519721077 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0f8406fbe0 0x7f0f8406fff0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0f74005fd0 tx=0x7f0f7402a4d0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.906+0000 7fda05569640 1 -- 172.21.15.37:0/2684871138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda0015a900 msgr2=0x7fda0006f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.906+0000 7fda05569640 1 --2- 172.21.15.37:0/2684871138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda0015a900 0x7fda0006f4f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fda000d1180 tx=0x7fd9f802a870 comp rx=0 tx=0).stop 2024-09-17T04:32:54.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.906+0000 7f0f8bc77640 1 -- 172.21.15.37:0/519721077 shutdown_connections 2024-09-17T04:32:54.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.906+0000 7f0f8bc77640 1 --2- 172.21.15.37:0/519721077 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0f8406fbe0 0x7f0f8406fff0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.906+0000 7f0f8bc77640 1 --2- 172.21.15.37:0/519721077 >> v2:172.21.15.37:3300/0 conn(0x7f0f840cdfb0 0x7f0f840cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.906+0000 7f0f8bc77640 1 -- 172.21.15.37:0/519721077 >> 172.21.15.37:0/519721077 conn(0x7f0f840d00b0 msgr2=0x7f0f840d04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:54.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.908+0000 7f6ae7fff640 1 -- 172.21.15.37:0/2675399468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ae808b910 msgr2=0x7f6ae80b3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.908+0000 7f6ae7fff640 1 --2- 172.21.15.37:0/2675399468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ae808b910 0x7f6ae80b3f90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8005b90 tx=0x7f6ad802b570 comp rx=0 tx=0).stop 2024-09-17T04:32:54.909 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi037') 2024-09-17T04:32:54.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.909+0000 7fc736a22640 1 -- 172.21.15.37:0/794474542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7300cf8d0 msgr2=0x7fc7300b3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.909+0000 7fc736a22640 1 --2- 172.21.15.37:0/794474542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7300cf8d0 0x7fc7300b3f90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc73006f2a0 tx=0x7fc72802ed10 comp rx=0 tx=0).stop 2024-09-17T04:32:54.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.909+0000 7f09dde0e640 1 -- 172.21.15.37:0/572527964 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09d80b4960 con 0x7f09d80b3aa0 2024-09-17T04:32:54.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.909+0000 7f6ae7fff640 1 -- 172.21.15.37:0/2675399468 shutdown_connections 2024-09-17T04:32:54.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.909+0000 7f6ae7fff640 1 --2- 172.21.15.37:0/2675399468 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6ae8072e90 0x7f6ae80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.909+0000 7f6ae7fff640 1 --2- 172.21.15.37:0/2675399468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ae808b910 0x7f6ae80b3f90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.910+0000 7f6ae7fff640 1 -- 172.21.15.37:0/2675399468 >> 172.21.15.37:0/2675399468 conn(0x7f6ae80b2000 msgr2=0x7f6ae80b2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:54.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.913+0000 7fda05569640 1 -- 172.21.15.37:0/2684871138 shutdown_connections 2024-09-17T04:32:54.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.913+0000 7fda05569640 1 --2- 172.21.15.37:0/2684871138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda0015a900 0x7fda0006f4f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.913+0000 7fda05569640 1 --2- 172.21.15.37:0/2684871138 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fda00159fb0 0x7fda0015a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.913+0000 7fda05569640 1 -- 172.21.15.37:0/2684871138 >> 172.21.15.37:0/2684871138 conn(0x7fda000d00d0 msgr2=0x7fda000d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:54.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.914+0000 7fc736a22640 1 -- 172.21.15.37:0/794474542 shutdown_connections 2024-09-17T04:32:54.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.914+0000 7fc736a22640 1 --2- 172.21.15.37:0/794474542 >> v2:172.21.15.37:3300/0 conn(0x7fc730072e90 0x7fc7300732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.914+0000 7fc736a22640 1 --2- 172.21.15.37:0/794474542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7300cf8d0 0x7fc7300b3f90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.914+0000 7fc736a22640 1 -- 172.21.15.37:0/794474542 >> 172.21.15.37:0/794474542 conn(0x7fc7300b2000 msgr2=0x7fc7300b2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:54.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.914+0000 7f09c3fff640 1 -- 172.21.15.37:0/572527964 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f09cc021020 con 0x7f09d80b3aa0 2024-09-17T04:32:54.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.913+0000 7f0f8bc77640 1 -- 172.21.15.37:0/519721077 shutdown_connections 2024-09-17T04:32:54.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.916+0000 7fc736a22640 1 -- 172.21.15.37:0/794474542 shutdown_connections 2024-09-17T04:32:54.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.910+0000 7f6ae7fff640 1 -- 172.21.15.37:0/2675399468 shutdown_connections 2024-09-17T04:32:54.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.910+0000 7f6ae7fff640 1 -- 172.21.15.37:0/2675399468 wait complete. 2024-09-17T04:32:54.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.913+0000 7f6ae7fff640 1 Processor -- start 2024-09-17T04:32:54.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.913+0000 7f6ae7fff640 1 -- start start 2024-09-17T04:32:54.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.915+0000 7f6ae7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ae8072e90 0x7f6ae8202df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.915+0000 7f6ae7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ae808b910 0x7f6ae8208dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.915+0000 7f6ae7fff640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6ae80cf1c0 0x7f6ae8205840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.915+0000 7f6ae7fff640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae8073960 con 0x7f6ae8072e90 2024-09-17T04:32:54.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.915+0000 7f6ae7fff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae80737e0 con 0x7f6ae808b910 2024-09-17T04:32:54.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.915+0000 7f6ae7fff640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f6ae8073ae0 con 0x7f6ae80cf1c0 2024-09-17T04:32:54.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.916+0000 7f6ae67fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ae808b910 0x7f6ae8208dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.916+0000 7f6ae67fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ae808b910 0x7f6ae8208dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.37:58378/0 (socket says 172.21.15.37:58378) 2024-09-17T04:32:54.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.916+0000 7f6ae67fc640 1 -- 172.21.15.37:0/945077455 learned_addr learned my addr 172.21.15.37:0/945077455 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:32:54.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.916+0000 7f6ae6ffd640 1 --2- 172.21.15.37:0/945077455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ae8072e90 0x7f6ae8202df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.916+0000 7f6ae67fc640 1 -- 172.21.15.37:0/945077455 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6ae80cf1c0 msgr2=0x7f6ae8205840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.916+0000 7f6ae67fc640 1 --2- 172.21.15.37:0/945077455 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6ae80cf1c0 0x7f6ae8205840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.916+0000 7f6ae67fc640 1 -- 172.21.15.37:0/945077455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ae8072e90 msgr2=0x7f6ae8202df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.916+0000 7f6ae67fc640 1 --2- 172.21.15.37:0/945077455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ae8072e90 0x7f6ae8202df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.916+0000 7f6ae67fc640 1 -- 172.21.15.37:0/945077455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae8202240 con 0x7f6ae808b910 2024-09-17T04:32:54.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.916+0000 7f6ae6ffd640 1 --2- 172.21.15.37:0/945077455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ae8072e90 0x7f6ae8202df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:32:54.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.917+0000 7f6ae67fc640 1 --2- 172.21.15.37:0/945077455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ae808b910 0x7f6ae8208dd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000cad0 tx=0x7f6ae000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:54.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.918+0000 7f6acf7fe640 1 -- 172.21.15.37:0/945077455 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0017070 con 0x7f6ae808b910 2024-09-17T04:32:54.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.918+0000 7f6ae7fff640 1 -- 172.21.15.37:0/945077455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae8204a80 con 0x7f6ae808b910 2024-09-17T04:32:54.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.918+0000 7f6ae7fff640 1 -- 172.21.15.37:0/945077455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae8205040 con 0x7f6ae808b910 2024-09-17T04:32:54.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.919+0000 7f0f8bc77640 1 -- 172.21.15.37:0/519721077 wait complete. 2024-09-17T04:32:54.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.916+0000 7fc736a22640 1 -- 172.21.15.37:0/794474542 wait complete. 2024-09-17T04:32:54.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.919+0000 7fc736a22640 1 Processor -- start 2024-09-17T04:32:54.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.920+0000 7f6acf7fe640 1 -- 172.21.15.37:0/945077455 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ae000a700 con 0x7f6ae808b910 2024-09-17T04:32:54.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.920+0000 7f6acf7fe640 1 -- 172.21.15.37:0/945077455 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae000ab10 con 0x7f6ae808b910 2024-09-17T04:32:54.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.920+0000 7f6acf7fe640 1 -- 172.21.15.37:0/945077455 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6ae001a070 con 0x7f6ae808b910 2024-09-17T04:32:54.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.920+0000 7f6acf7fe640 1 -- 172.21.15.37:0/945077455 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6ae001c860 con 0x7f6ae808b910 2024-09-17T04:32:54.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.921+0000 7fda05569640 1 -- 172.21.15.37:0/2684871138 shutdown_connections 2024-09-17T04:32:54.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.921+0000 7fda05569640 1 -- 172.21.15.37:0/2684871138 wait complete. 2024-09-17T04:32:54.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.920+0000 7fc736a22640 1 -- start start 2024-09-17T04:32:54.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.922+0000 7f6acd7fa640 1 -- 172.21.15.37:0/945077455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ac8005290 con 0x7f6ae808b910 2024-09-17T04:32:54.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.922+0000 7f0f8bc77640 1 Processor -- start 2024-09-17T04:32:54.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.922+0000 7f0f8bc77640 1 -- start start 2024-09-17T04:32:54.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.924+0000 7f0f8bc77640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f8406fbe0 0x7f0f8415b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.924+0000 7f0f8bc77640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0f840cb180 0x7f0f8415f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.924+0000 7f0f8bc77640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f840cdfb0 0x7f0f8415f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.924+0000 7f0f8bc77640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0f84162980 con 0x7f0f840cdfb0 2024-09-17T04:32:54.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.924+0000 7f0f8bc77640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0f84162800 con 0x7f0f8406fbe0 2024-09-17T04:32:54.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.924+0000 7f0f8bc77640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f0f84162b00 con 0x7f0f840cb180 2024-09-17T04:32:54.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.925+0000 7f6acf7fe640 1 -- 172.21.15.37:0/945077455 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f6ae0021020 con 0x7f6ae808b910 2024-09-17T04:32:54.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.922+0000 7fc736a22640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc730072e90 0x7fc730213200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.927+0000 7fc735a20640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc730072e90 0x7fc730213200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.927+0000 7fc735a20640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc730072e90 0x7fc730213200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56686/0 (socket says 172.21.15.37:56686) 2024-09-17T04:32:54.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.927+0000 7fc735a20640 1 -- 172.21.15.37:0/2087360365 learned_addr learned my addr 172.21.15.37:0/2087360365 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:32:54.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.928+0000 7fc736a22640 1 --2- 172.21.15.37:0/2087360365 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc73008b910 0x7fc7300895f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.928+0000 7fc736a22640 1 --2- 172.21.15.37:0/2087360365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc73008a1b0 0x7fc730089b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.928+0000 7fc73521f640 1 --2- 172.21.15.37:0/2087360365 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc73008b910 0x7fc7300895f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.928+0000 7fc736a22640 1 -- 172.21.15.37:0/2087360365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc730073960 con 0x7fc730072e90 2024-09-17T04:32:54.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.928+0000 7fc736221640 1 --2- 172.21.15.37:0/2087360365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc73008a1b0 0x7fc730089b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.925+0000 7f0f8a214640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f840cdfb0 0x7f0f8415f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.925+0000 7f0f8a214640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f840cdfb0 0x7f0f8415f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56670/0 (socket says 172.21.15.37:56670) 2024-09-17T04:32:54.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.925+0000 7f0f8a214640 1 -- 172.21.15.37:0/2849547655 learned_addr learned my addr 172.21.15.37:0/2849547655 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:32:54.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.925+0000 7f0f89a13640 1 --2- 172.21.15.37:0/2849547655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f8406fbe0 0x7f0f8415b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.925+0000 7f0f89212640 1 --2- 172.21.15.37:0/2849547655 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0f840cb180 0x7f0f8415f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.925+0000 7f0f8a214640 1 -- 172.21.15.37:0/2849547655 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0f840cb180 msgr2=0x7f0f8415f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.925+0000 7f0f8a214640 1 --2- 172.21.15.37:0/2849547655 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0f840cb180 0x7f0f8415f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.926+0000 7f0f8a214640 1 -- 172.21.15.37:0/2849547655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f8406fbe0 msgr2=0x7f0f8415b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.926+0000 7f0f8a214640 1 --2- 172.21.15.37:0/2849547655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f8406fbe0 0x7f0f8415b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.926+0000 7f0f8a214640 1 -- 172.21.15.37:0/2849547655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f840c6bc0 con 0x7f0f840cdfb0 2024-09-17T04:32:54.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.926+0000 7f0f8a214640 1 --2- 172.21.15.37:0/2849547655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f840cdfb0 0x7f0f8415f620 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0f740099f0 tx=0x7f0f74004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:54.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.926+0000 7f0f7a7fc640 1 -- 172.21.15.37:0/2849547655 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f74041070 con 0x7f0f840cdfb0 2024-09-17T04:32:54.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.927+0000 7f0f7a7fc640 1 -- 172.21.15.37:0/2849547655 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f740038e0 con 0x7f0f840cdfb0 2024-09-17T04:32:54.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.927+0000 7f0f7a7fc640 1 -- 172.21.15.37:0/2849547655 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f74003cf0 con 0x7f0f840cdfb0 2024-09-17T04:32:54.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.927+0000 7f0f8bc77640 1 -- 172.21.15.37:0/2849547655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f840b4490 con 0x7f0f840cdfb0 2024-09-17T04:32:54.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.927+0000 7f0f8bc77640 1 -- 172.21.15.37:0/2849547655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f840b4a50 con 0x7f0f840cdfb0 2024-09-17T04:32:54.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.928+0000 7f0f89a13640 1 --2- 172.21.15.37:0/2849547655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f8406fbe0 0x7f0f8415b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:54.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.928+0000 7f0f7a7fc640 1 -- 172.21.15.37:0/2849547655 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0f74044070 con 0x7f0f840cdfb0 2024-09-17T04:32:54.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.928+0000 7f0f7a7fc640 1 -- 172.21.15.37:0/2849547655 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0f74046c30 con 0x7f0f840cdfb0 2024-09-17T04:32:54.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.928+0000 7f0f89212640 1 --2- 172.21.15.37:0/2849547655 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0f840cb180 0x7f0f8415f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:32:54.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.929+0000 7fc736a22640 1 -- 172.21.15.37:0/2087360365 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc7300737e0 con 0x7fc73008a1b0 2024-09-17T04:32:54.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.929+0000 7fc736a22640 1 -- 172.21.15.37:0/2087360365 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fc730073ae0 con 0x7fc73008b910 2024-09-17T04:32:54.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.930+0000 7fda05569640 1 Processor -- start 2024-09-17T04:32:54.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.929+0000 7fc735a20640 1 -- 172.21.15.37:0/2087360365 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc73008b910 msgr2=0x7fc7300895f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.930+0000 7fda05569640 1 -- start start 2024-09-17T04:32:54.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.929+0000 7fc735a20640 1 --2- 172.21.15.37:0/2087360365 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc73008b910 0x7fc7300895f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.929+0000 7fc735a20640 1 -- 172.21.15.37:0/2087360365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc73008a1b0 msgr2=0x7fc730089b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.929+0000 7fc735a20640 1 --2- 172.21.15.37:0/2087360365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc73008a1b0 0x7fc730089b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.929+0000 7fc735a20640 1 -- 172.21.15.37:0/2087360365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc730213ec0 con 0x7fc730072e90 2024-09-17T04:32:54.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.930+0000 7fc735a20640 1 --2- 172.21.15.37:0/2087360365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc730072e90 0x7fc730213200 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc72000c730 tx=0x7fc72000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:54.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.930+0000 7fc73521f640 1 --2- 172.21.15.37:0/2087360365 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc73008b910 0x7fc7300895f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:32:54.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.931+0000 7fc7267fc640 1 -- 172.21.15.37:0/2087360365 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7200052d0 con 0x7fc730072e90 2024-09-17T04:32:54.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.931+0000 7fc7267fc640 1 -- 172.21.15.37:0/2087360365 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc720005470 con 0x7fc730072e90 2024-09-17T04:32:54.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.931+0000 7fc7267fc640 1 -- 172.21.15.37:0/2087360365 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72000a880 con 0x7fc730072e90 2024-09-17T04:32:54.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.931+0000 7fc736221640 1 --2- 172.21.15.37:0/2087360365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc73008a1b0 0x7fc730089b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:32:54.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.931+0000 7fc736a22640 1 -- 172.21.15.37:0/2087360365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7300c4f20 con 0x7fc730072e90 2024-09-17T04:32:54.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.933+0000 7fc736a22640 1 -- 172.21.15.37:0/2087360365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7301f4b50 con 0x7fc730072e90 2024-09-17T04:32:54.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.933+0000 7fda05569640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda000d0bf0 0x7fda000b5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.933+0000 7fda05569640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda00159fb0 0x7fda00140260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.934+0000 7fc7267fc640 1 -- 172.21.15.37:0/2087360365 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc72000aa20 con 0x7fc730072e90 2024-09-17T04:32:54.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.935+0000 7fc7267fc640 1 -- 172.21.15.37:0/2087360365 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc720012820 con 0x7fc730072e90 2024-09-17T04:32:54.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.935+0000 7fc713fff640 1 -- 172.21.15.37:0/2087360365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc730073f90 con 0x7fc730072e90 2024-09-17T04:32:54.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.938+0000 7f0f8bc77640 1 -- 172.21.15.37:0/2849547655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f6c005290 con 0x7f0f840cdfb0 2024-09-17T04:32:54.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.939+0000 7fda05569640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fda0015a900 0x7fda001407a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:54.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.939+0000 7fda05569640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fda00073b10 con 0x7fda000d0bf0 2024-09-17T04:32:54.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.939+0000 7fda05569640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fda00073990 con 0x7fda00159fb0 2024-09-17T04:32:54.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.939+0000 7fda05569640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fda0006fa30 con 0x7fda0015a900 2024-09-17T04:32:54.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.943+0000 7fc7267fc640 1 -- 172.21.15.37:0/2087360365 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fc720020020 con 0x7fc730072e90 2024-09-17T04:32:54.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.943+0000 7fd9ff7fe640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fda0015a900 0x7fda001407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.943+0000 7fd9ff7fe640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fda0015a900 0x7fda001407a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.37:40550/0 (socket says 172.21.15.37:40550) 2024-09-17T04:32:54.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.943+0000 7fd9ff7fe640 1 -- 172.21.15.37:0/4228345858 learned_addr learned my addr 172.21.15.37:0/4228345858 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:32:54.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.943+0000 7f0f7a7fc640 1 -- 172.21.15.37:0/2849547655 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0f840cb590 con 0x7f0f840cdfb0 2024-09-17T04:32:54.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.943+0000 7fd9fe7fc640 1 --2- 172.21.15.37:0/4228345858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda00159fb0 0x7fda00140260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.943+0000 7fd9ff7fe640 1 -- 172.21.15.37:0/4228345858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda00159fb0 msgr2=0x7fda00140260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.943+0000 7fd9ff7fe640 1 --2- 172.21.15.37:0/4228345858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda00159fb0 0x7fda00140260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.943+0000 7fd9feffd640 1 --2- 172.21.15.37:0/4228345858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda000d0bf0 0x7fda000b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:54.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.943+0000 7fd9ff7fe640 1 -- 172.21.15.37:0/4228345858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda000d0bf0 msgr2=0x7fda000b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:54.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.943+0000 7fd9ff7fe640 1 --2- 172.21.15.37:0/4228345858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda000d0bf0 0x7fda000b5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:54.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.943+0000 7fd9ff7fe640 1 -- 172.21.15.37:0/4228345858 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda000b42d0 con 0x7fda0015a900 2024-09-17T04:32:54.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.943+0000 7fd9feffd640 1 --2- 172.21.15.37:0/4228345858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda000d0bf0 0x7fda000b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:32:54.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.943+0000 7fd9ff7fe640 1 --2- 172.21.15.37:0/4228345858 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fda0015a900 0x7fda001407a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd9f802ae60 tx=0x7fd9f8003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.944+0000 7fd9e77fe640 1 -- 172.21.15.37:0/4228345858 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8031b50 con 0x7fda0015a900 2024-09-17T04:32:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.944+0000 7fd9e77fe640 1 -- 172.21.15.37:0/4228345858 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9f8043070 con 0x7fda0015a900 2024-09-17T04:32:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.945+0000 7fd9e77fe640 1 -- 172.21.15.37:0/4228345858 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f80330d0 con 0x7fda0015a900 2024-09-17T04:32:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.946+0000 7fda05569640 1 -- 172.21.15.37:0/4228345858 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda000b45b0 con 0x7fda0015a900 2024-09-17T04:32:54.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.946+0000 7fd9e77fe640 1 -- 172.21.15.37:0/4228345858 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd9f8030070 con 0x7fda0015a900 2024-09-17T04:32:54.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.946+0000 7fda05569640 1 -- 172.21.15.37:0/4228345858 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fda000b4b70 con 0x7fda0015a900 2024-09-17T04:32:54.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.947+0000 7fd9e77fe640 1 -- 172.21.15.37:0/4228345858 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd9f80040a0 con 0x7fda0015a900 2024-09-17T04:32:54.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.953+0000 7fd9e57fa640 1 -- 172.21.15.37:0/4228345858 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9d8005290 con 0x7fda0015a900 2024-09-17T04:32:54.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.961+0000 7fd9e77fe640 1 -- 172.21.15.37:0/4228345858 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd9f804d020 con 0x7fda0015a900 2024-09-17T04:32:55.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.000+0000 7f87508b6640 1 -- 172.21.15.37:0/3065825660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f87440e6a20 con 0x7f87440544d0 2024-09-17T04:32:55.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:54.996+0000 7ff9ba69b640 1 -- 172.21.15.37:0/3058839357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7ff9ac054a50 con 0x7ff9ac051be0 2024-09-17T04:32:55.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.006+0000 7ff9b0ff9640 1 -- 172.21.15.37:0/3058839357 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v11)=0 v11) ==== 119+0+0 (secure 0 0 0) 0x7ff9a000ce00 con 0x7ff9ac051be0 2024-09-17T04:32:55.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.007+0000 7ff9ba69b640 1 -- 172.21.15.37:0/3058839357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9ac051be0 msgr2=0x7ff9ac0e0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.007+0000 7ff9ba69b640 1 --2- 172.21.15.37:0/3058839357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9ac051be0 0x7ff9ac0e0680 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000b780 tx=0x7ff9a000bc40 comp rx=0 tx=0).stop 2024-09-17T04:32:55.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.007+0000 7ff9ba69b640 1 -- 172.21.15.37:0/3058839357 shutdown_connections 2024-09-17T04:32:55.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.007+0000 7ff9ba69b640 1 --2- 172.21.15.37:0/3058839357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9ac0e0bc0 0x7ff9ac0321e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.007+0000 7ff9ba69b640 1 --2- 172.21.15.37:0/3058839357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9ac051be0 0x7ff9ac0e0680 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.007+0000 7ff9ba69b640 1 --2- 172.21.15.37:0/3058839357 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff9ac006070 0x7ff9ac0dc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.007+0000 7ff9ba69b640 1 -- 172.21.15.37:0/3058839357 >> 172.21.15.37:0/3058839357 conn(0x7ff9ac01e230 msgr2=0x7ff9ac039ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:55.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.008+0000 7ff9ba69b640 1 -- 172.21.15.37:0/3058839357 shutdown_connections 2024-09-17T04:32:55.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.008+0000 7ff9ba69b640 1 -- 172.21.15.37:0/3058839357 wait complete. 2024-09-17T04:32:55.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.012+0000 7f873affd640 1 -- 172.21.15.37:0/3065825660 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f873c042790 con 0x7f87440544d0 2024-09-17T04:32:55.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.013+0000 7f8738ff9640 1 -- 172.21.15.37:0/3065825660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87440544d0 msgr2=0x7f874404bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.013+0000 7f8738ff9640 1 --2- 172.21.15.37:0/3065825660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87440544d0 0x7f874404bbd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f873c044040 tx=0x7f873c004340 comp rx=0 tx=0).stop 2024-09-17T04:32:55.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.014+0000 7f8738ff9640 1 -- 172.21.15.37:0/3065825660 shutdown_connections 2024-09-17T04:32:55.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.014+0000 7f8738ff9640 1 --2- 172.21.15.37:0/3065825660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87440544d0 0x7f874404bbd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.014+0000 7f8738ff9640 1 --2- 172.21.15.37:0/3065825660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87440538c0 0x7f874404b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.014+0000 7f8738ff9640 1 --2- 172.21.15.37:0/3065825660 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8744037490 0x7f874404c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.014+0000 7f8738ff9640 1 -- 172.21.15.37:0/3065825660 >> 172.21.15.37:0/3065825660 conn(0x7f8744055e70 msgr2=0x7f87440e1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:55.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.017+0000 7fa944e30640 1 -- 172.21.15.37:0/2293969086 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7fa924005850 con 0x7fa94006fd00 2024-09-17T04:32:55.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.025+0000 7f8738ff9640 1 -- 172.21.15.37:0/3065825660 shutdown_connections 2024-09-17T04:32:55.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.025+0000 7f8738ff9640 1 -- 172.21.15.37:0/3065825660 wait complete. 2024-09-17T04:32:55.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.031+0000 7f4d53fff640 1 -- 172.21.15.37:0/2506532410 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f4d48005850 con 0x7f4d6806fd00 2024-09-17T04:32:55.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.038+0000 7fa92effd640 1 -- 172.21.15.37:0/2293969086 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v13)=0 v13) ==== 119+0+0 (secure 0 0 0) 0x7fa9300047a0 con 0x7fa94006fd00 2024-09-17T04:32:55.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.040+0000 7fa92cff9640 1 -- 172.21.15.37:0/2293969086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa94006fd00 msgr2=0x7fa940168100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.040+0000 7fa92cff9640 1 --2- 172.21.15.37:0/2293969086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa94006fd00 0x7fa940168100 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa930009770 tx=0x7fa93000e390 comp rx=0 tx=0).stop 2024-09-17T04:32:55.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.043+0000 7fa92cff9640 1 -- 172.21.15.37:0/2293969086 shutdown_connections 2024-09-17T04:32:55.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.043+0000 7fa92cff9640 1 --2- 172.21.15.37:0/2293969086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa94015bf90 0x7fa94015c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.043+0000 7fa92cff9640 1 --2- 172.21.15.37:0/2293969086 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa94016b210 0x7fa940169ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.043+0000 7fa92cff9640 1 --2- 172.21.15.37:0/2293969086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa94006fd00 0x7fa940168100 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.043+0000 7fa92cff9640 1 -- 172.21.15.37:0/2293969086 >> 172.21.15.37:0/2293969086 conn(0x7fa94006d390 msgr2=0x7fa94016d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:55.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.043+0000 7fa92cff9640 1 -- 172.21.15.37:0/2293969086 shutdown_connections 2024-09-17T04:32:55.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.043+0000 7fa92cff9640 1 -- 172.21.15.37:0/2293969086 wait complete. 2024-09-17T04:32:55.048 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi037') 2024-09-17T04:32:55.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.050+0000 7f4d5e7fc640 1 -- 172.21.15.37:0/2506532410 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v14)=0 v14) ==== 121+0+0 (secure 0 0 0) 0x7f4d600045c0 con 0x7f4d6806fd00 2024-09-17T04:32:55.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.051+0000 7f4d53fff640 1 -- 172.21.15.37:0/2506532410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d6806fd00 msgr2=0x7f4d6815afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.051+0000 7f4d53fff640 1 --2- 172.21.15.37:0/2506532410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d6806fd00 0x7f4d6815afe0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4d60033040 tx=0x7f4d60003f50 comp rx=0 tx=0).stop 2024-09-17T04:32:55.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.051+0000 7f4d53fff640 1 -- 172.21.15.37:0/2506532410 shutdown_connections 2024-09-17T04:32:55.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.051+0000 7f4d53fff640 1 --2- 172.21.15.37:0/2506532410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d6815b7e0 0x7f4d6817cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.051+0000 7f4d53fff640 1 --2- 172.21.15.37:0/2506532410 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4d6816b210 0x7f4d6817c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.051+0000 7f4d53fff640 1 --2- 172.21.15.37:0/2506532410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d6806fd00 0x7f4d6815afe0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.051+0000 7f4d53fff640 1 -- 172.21.15.37:0/2506532410 >> 172.21.15.37:0/2506532410 conn(0x7f4d6806d390 msgr2=0x7f4d680b4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:55.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.051+0000 7f4d53fff640 1 -- 172.21.15.37:0/2506532410 shutdown_connections 2024-09-17T04:32:55.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.052+0000 7f4d53fff640 1 -- 172.21.15.37:0/2506532410 wait complete. 2024-09-17T04:32:55.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.059+0000 7fdba77fe640 1 -- 172.21.15.37:0/1556303495 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fdbb4054990 con 0x7fdbb404b870 2024-09-17T04:32:55.071 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi037') 2024-09-17T04:32:55.072 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi037') 2024-09-17T04:32:55.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.077+0000 7fdbb1ffb640 1 -- 172.21.15.37:0/1556303495 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v15)=0 v15) ==== 121+0+0 (secure 0 0 0) 0x7fdbb8007d20 con 0x7fdbb404b870 2024-09-17T04:32:55.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.078+0000 7fdba77fe640 1 -- 172.21.15.37:0/1556303495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbb404b870 msgr2=0x7fdbb404df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.078+0000 7fdba77fe640 1 --2- 172.21.15.37:0/1556303495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbb404b870 0x7fdbb404df60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4037b50 tx=0x7fdbb800ad00 comp rx=0 tx=0).stop 2024-09-17T04:32:55.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.081+0000 7fdba77fe640 1 -- 172.21.15.37:0/1556303495 shutdown_connections 2024-09-17T04:32:55.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.081+0000 7fdba77fe640 1 --2- 172.21.15.37:0/1556303495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdbb404b870 0x7fdbb404df60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.081+0000 7fdba77fe640 1 --2- 172.21.15.37:0/1556303495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbb404b460 0x7fdbb404da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.081+0000 7fdba77fe640 1 --2- 172.21.15.37:0/1556303495 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdba8003a00 0x7fdbb4187220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.081+0000 7fdba77fe640 1 -- 172.21.15.37:0/1556303495 >> 172.21.15.37:0/1556303495 conn(0x7fdbb401e230 msgr2=0x7fdbb4005000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:55.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.088+0000 7fdba77fe640 1 -- 172.21.15.37:0/1556303495 shutdown_connections 2024-09-17T04:32:55.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.088+0000 7fdba77fe640 1 -- 172.21.15.37:0/1556303495 wait complete. 2024-09-17T04:32:55.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.088+0000 7fc713fff640 1 -- 172.21.15.37:0/2087360365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7fc73016abb0 con 0x7fc730072e90 2024-09-17T04:32:55.104 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi037') 2024-09-17T04:32:55.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.098+0000 7fd36e7fc640 1 -- 172.21.15.37:0/2993994265 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fd368005580 con 0x7fd370003ae0 2024-09-17T04:32:55.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.107+0000 7f6dadffb640 1 -- 172.21.15.37:0/3830474725 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f6da8005850 con 0x7f6dc80b31a0 2024-09-17T04:32:55.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.108+0000 7fc7267fc640 1 -- 172.21.15.37:0/2087360365 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v16)=0 v16) ==== 125+0+0 (secure 0 0 0) 0x7fc72000ad10 con 0x7fc730072e90 2024-09-17T04:32:55.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.109+0000 7fc736a22640 1 -- 172.21.15.37:0/2087360365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc730072e90 msgr2=0x7fc730213200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.109+0000 7fc736a22640 1 --2- 172.21.15.37:0/2087360365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc730072e90 0x7fc730213200 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc72000c730 tx=0x7fc72000cbf0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.109+0000 7fc736a22640 1 -- 172.21.15.37:0/2087360365 shutdown_connections 2024-09-17T04:32:55.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.109+0000 7fc736a22640 1 --2- 172.21.15.37:0/2087360365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc73008a1b0 0x7fc730089b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.109+0000 7fc736a22640 1 --2- 172.21.15.37:0/2087360365 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc73008b910 0x7fc7300895f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.109+0000 7fc736a22640 1 --2- 172.21.15.37:0/2087360365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc730072e90 0x7fc730213200 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.109+0000 7fc736a22640 1 -- 172.21.15.37:0/2087360365 >> 172.21.15.37:0/2087360365 conn(0x7fc7300b2000 msgr2=0x7fc73016e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:55.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.109+0000 7fc736a22640 1 -- 172.21.15.37:0/2087360365 shutdown_connections 2024-09-17T04:32:55.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.109+0000 7fc736a22640 1 -- 172.21.15.37:0/2087360365 wait complete. 2024-09-17T04:32:55.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.113+0000 7fd384ff9640 1 -- 172.21.15.37:0/2993994265 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v17)=0 v17) ==== 120+0+0 (secure 0 0 0) 0x7fd380040af0 con 0x7fd370003ae0 2024-09-17T04:32:55.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.119+0000 7f6daffff640 1 -- 172.21.15.37:0/3830474725 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v18)=0 v18) ==== 120+0+0 (secure 0 0 0) 0x7f6dc8163f00 con 0x7f6dc80b31a0 2024-09-17T04:32:55.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.115+0000 7fd36e7fc640 1 -- 172.21.15.37:0/2993994265 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd370003ae0 msgr2=0x7fd3881e6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.115+0000 7fd36e7fc640 1 --2- 172.21.15.37:0/2993994265 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd370003ae0 0x7fd3881e6df0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd380000f80 tx=0x7fd380001aa0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.115+0000 7fd36e7fc640 1 -- 172.21.15.37:0/2993994265 shutdown_connections 2024-09-17T04:32:55.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.115+0000 7fd36e7fc640 1 --2- 172.21.15.37:0/2993994265 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3880b20c0 0x7fd3881f82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.115+0000 7fd36e7fc640 1 --2- 172.21.15.37:0/2993994265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3881e7720 0x7fd3880b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.115+0000 7fd36e7fc640 1 --2- 172.21.15.37:0/2993994265 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd370003ae0 0x7fd3881e6df0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.115+0000 7fd36e7fc640 1 -- 172.21.15.37:0/2993994265 >> 172.21.15.37:0/2993994265 conn(0x7fd38809d800 msgr2=0x7fd3880b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:55.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.116+0000 7fd36e7fc640 1 -- 172.21.15.37:0/2993994265 shutdown_connections 2024-09-17T04:32:55.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.116+0000 7fd36e7fc640 1 -- 172.21.15.37:0/2993994265 wait complete. 2024-09-17T04:32:55.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.127+0000 7f6dadffb640 1 -- 172.21.15.37:0/3830474725 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6dc80b31a0 msgr2=0x7f6dc80ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.127+0000 7f6dadffb640 1 --2- 172.21.15.37:0/3830474725 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6dc80b31a0 0x7f6dc80ceaa0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000cbc0 tx=0x7f6dc000a430 comp rx=0 tx=0).stop 2024-09-17T04:32:55.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.127+0000 7f6dadffb640 1 -- 172.21.15.37:0/3830474725 shutdown_connections 2024-09-17T04:32:55.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.127+0000 7f6dadffb640 1 --2- 172.21.15.37:0/3830474725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dc816c3e0 0x7f6dc80cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.127+0000 7f6dadffb640 1 --2- 172.21.15.37:0/3830474725 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6dc80b31a0 0x7f6dc80ceaa0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.127+0000 7f6dadffb640 1 --2- 172.21.15.37:0/3830474725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dc8070410 0x7f6dc81633c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.127+0000 7f6dadffb640 1 -- 172.21.15.37:0/3830474725 >> 172.21.15.37:0/3830474725 conn(0x7f6dc806d390 msgr2=0x7f6dc80b1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:55.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.127+0000 7f6dadffb640 1 -- 172.21.15.37:0/3830474725 shutdown_connections 2024-09-17T04:32:55.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.127+0000 7f6dadffb640 1 -- 172.21.15.37:0/3830474725 wait complete. 2024-09-17T04:32:55.137 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi037') 2024-09-17T04:32:55.145 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi037') 2024-09-17T04:32:55.145 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi037') 2024-09-17T04:32:55.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.154+0000 7fb044d7b640 1 -- 172.21.15.37:0/3104482230 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7fb024005850 con 0x7fb0401621b0 2024-09-17T04:32:55.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.158+0000 7fb02effd640 1 -- 172.21.15.37:0/3104482230 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v19)=0 v19) ==== 108+0+0 (secure 0 0 0) 0x7fb03801e510 con 0x7fb0401621b0 2024-09-17T04:32:55.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.159+0000 7f09dde0e640 1 -- 172.21.15.37:0/572527964 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f09d815ecf0 con 0x7f09d80b3aa0 2024-09-17T04:32:55.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.159+0000 7f6acd7fa640 1 -- 172.21.15.37:0/945077455 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f6ac8005580 con 0x7f6ae808b910 2024-09-17T04:32:55.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.159+0000 7fd9e57fa640 1 -- 172.21.15.37:0/4228345858 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fd9d8005850 con 0x7fda0015a900 2024-09-17T04:32:55.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.160+0000 7fb044d7b640 1 -- 172.21.15.37:0/3104482230 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0401621b0 msgr2=0x7fb040153ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.160+0000 7fb044d7b640 1 --2- 172.21.15.37:0/3104482230 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0401621b0 0x7fb040153ae0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb03800b9c0 tx=0x7fb03800be80 comp rx=0 tx=0).stop 2024-09-17T04:32:55.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.160+0000 7fb044d7b640 1 -- 172.21.15.37:0/3104482230 shutdown_connections 2024-09-17T04:32:55.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.160+0000 7fb044d7b640 1 --2- 172.21.15.37:0/3104482230 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0401621b0 0x7fb040153ae0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.160+0000 7fb044d7b640 1 --2- 172.21.15.37:0/3104482230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0400b3790 0x7fb0400acbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.160+0000 7fb044d7b640 1 --2- 172.21.15.37:0/3104482230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0400745e0 0x7fb0401fa3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.160+0000 7fb044d7b640 1 -- 172.21.15.37:0/3104482230 >> 172.21.15.37:0/3104482230 conn(0x7fb04006d390 msgr2=0x7fb040160780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:55.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.160+0000 7fb044d7b640 1 -- 172.21.15.37:0/3104482230 shutdown_connections 2024-09-17T04:32:55.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.160+0000 7fb044d7b640 1 -- 172.21.15.37:0/3104482230 wait complete. 2024-09-17T04:32:55.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.162+0000 7f09c3fff640 1 -- 172.21.15.37:0/572527964 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v20)=0 v20) ==== 106+0+0 (secure 0 0 0) 0x7f09cc00ad80 con 0x7f09d80b3aa0 2024-09-17T04:32:55.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.163+0000 7f09dde0e640 1 -- 172.21.15.37:0/572527964 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f09d80b3aa0 msgr2=0x7f09d815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.163+0000 7f09dde0e640 1 --2- 172.21.15.37:0/572527964 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f09d80b3aa0 0x7f09d815d4c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00c730 tx=0x7f09cc00cbf0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.163+0000 7f09dde0e640 1 -- 172.21.15.37:0/572527964 shutdown_connections 2024-09-17T04:32:55.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.163+0000 7f09dde0e640 1 --2- 172.21.15.37:0/572527964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09d815da00 0x7f09d8154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.163+0000 7f09dde0e640 1 --2- 172.21.15.37:0/572527964 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f09d80b3aa0 0x7f09d815d4c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.163+0000 7f09dde0e640 1 --2- 172.21.15.37:0/572527964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09d8070c80 0x7f09d8142360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.163+0000 7f09dde0e640 1 -- 172.21.15.37:0/572527964 >> 172.21.15.37:0/572527964 conn(0x7f09d80b1c10 msgr2=0x7f09d80b0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:55.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.163+0000 7f09dde0e640 1 -- 172.21.15.37:0/572527964 shutdown_connections 2024-09-17T04:32:55.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.163+0000 7f09dde0e640 1 -- 172.21.15.37:0/572527964 wait complete. 2024-09-17T04:32:55.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.165+0000 7f6acf7fe640 1 -- 172.21.15.37:0/945077455 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v21)=0 v21) ==== 111+0+0 (secure 0 0 0) 0x7f6ae000ae00 con 0x7f6ae808b910 2024-09-17T04:32:55.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.166+0000 7f6acd7fa640 1 -- 172.21.15.37:0/945077455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ae808b910 msgr2=0x7f6ae8208dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.166+0000 7f6acd7fa640 1 --2- 172.21.15.37:0/945077455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ae808b910 0x7f6ae8208dd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000cad0 tx=0x7f6ae000cf90 comp rx=0 tx=0).stop 2024-09-17T04:32:55.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.166+0000 7f6acd7fa640 1 -- 172.21.15.37:0/945077455 shutdown_connections 2024-09-17T04:32:55.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.167+0000 7f6acd7fa640 1 --2- 172.21.15.37:0/945077455 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6ae80cf1c0 0x7f6ae8205840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.167+0000 7f6acd7fa640 1 --2- 172.21.15.37:0/945077455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ae808b910 0x7f6ae8208dd0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.167+0000 7f6acd7fa640 1 --2- 172.21.15.37:0/945077455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ae8072e90 0x7f6ae8202df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.167+0000 7f6acd7fa640 1 -- 172.21.15.37:0/945077455 >> 172.21.15.37:0/945077455 conn(0x7f6ae80b2000 msgr2=0x7f6ae816d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:55.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.167+0000 7f6acd7fa640 1 -- 172.21.15.37:0/945077455 shutdown_connections 2024-09-17T04:32:55.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.167+0000 7f6acd7fa640 1 -- 172.21.15.37:0/945077455 wait complete. 2024-09-17T04:32:55.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.169+0000 7fd9e77fe640 1 -- 172.21.15.37:0/4228345858 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v22)=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7fd9f80044f0 con 0x7fda0015a900 2024-09-17T04:32:55.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.171+0000 7fd9e57fa640 1 -- 172.21.15.37:0/4228345858 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fda0015a900 msgr2=0x7fda001407a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.171+0000 7fd9e57fa640 1 --2- 172.21.15.37:0/4228345858 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fda0015a900 0x7fda001407a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd9f802ae60 tx=0x7fd9f8003ba0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.171+0000 7fd9e57fa640 1 -- 172.21.15.37:0/4228345858 shutdown_connections 2024-09-17T04:32:55.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.171+0000 7fd9e57fa640 1 --2- 172.21.15.37:0/4228345858 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fda0015a900 0x7fda001407a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.171+0000 7fd9e57fa640 1 --2- 172.21.15.37:0/4228345858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda00159fb0 0x7fda00140260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.171+0000 7fd9e57fa640 1 --2- 172.21.15.37:0/4228345858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda000d0bf0 0x7fda000b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.171+0000 7fd9e57fa640 1 -- 172.21.15.37:0/4228345858 >> 172.21.15.37:0/4228345858 conn(0x7fda000d00d0 msgr2=0x7fda000b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:55.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.171+0000 7fd9e57fa640 1 -- 172.21.15.37:0/4228345858 shutdown_connections 2024-09-17T04:32:55.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.171+0000 7fd9e57fa640 1 -- 172.21.15.37:0/4228345858 wait complete. 2024-09-17T04:32:55.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.176+0000 7f0f8bc77640 1 -- 172.21.15.37:0/2849547655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f0f6c005580 con 0x7f0f840cdfb0 2024-09-17T04:32:55.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.178+0000 7f0f7a7fc640 1 -- 172.21.15.37:0/2849547655 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v23)=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f0f74046450 con 0x7f0f840cdfb0 2024-09-17T04:32:55.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.180+0000 7f0f8bc77640 1 -- 172.21.15.37:0/2849547655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f840cdfb0 msgr2=0x7f0f8415f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.180+0000 7f0f8bc77640 1 --2- 172.21.15.37:0/2849547655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f840cdfb0 0x7f0f8415f620 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0f740099f0 tx=0x7f0f74004260 comp rx=0 tx=0).stop 2024-09-17T04:32:55.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.180+0000 7f0f8bc77640 1 -- 172.21.15.37:0/2849547655 shutdown_connections 2024-09-17T04:32:55.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.180+0000 7f0f8bc77640 1 --2- 172.21.15.37:0/2849547655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f840cdfb0 0x7f0f8415f620 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.180+0000 7f0f8bc77640 1 --2- 172.21.15.37:0/2849547655 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0f840cb180 0x7f0f8415f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.180+0000 7f0f8bc77640 1 --2- 172.21.15.37:0/2849547655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f8406fbe0 0x7f0f8415b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.180+0000 7f0f8bc77640 1 -- 172.21.15.37:0/2849547655 >> 172.21.15.37:0/2849547655 conn(0x7f0f840d00b0 msgr2=0x7f0f840b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:55.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.180+0000 7f0f8bc77640 1 -- 172.21.15.37:0/2849547655 shutdown_connections 2024-09-17T04:32:55.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.180+0000 7f0f8bc77640 1 -- 172.21.15.37:0/2849547655 wait complete. 2024-09-17T04:32:55.181 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi037') 2024-09-17T04:32:55.181 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi037') 2024-09-17T04:32:55.182 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi037') 2024-09-17T04:32:55.182 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi037') 2024-09-17T04:32:55.191 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi037') 2024-09-17T04:32:55.191 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config dump 2024-09-17T04:32:55.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.346+0000 7fba7cee5640 1 -- 172.21.15.37:0/923178886 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fba780b7af0 msgr2=0x7fba780ced00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.346+0000 7fba7cee5640 1 --2- 172.21.15.37:0/923178886 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fba780b7af0 0x7fba780ced00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fba68009830 tx=0x7fba6802a510 comp rx=0 tx=0).stop 2024-09-17T04:32:55.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.346+0000 7fba7cee5640 1 -- 172.21.15.37:0/923178886 shutdown_connections 2024-09-17T04:32:55.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.346+0000 7fba7cee5640 1 --2- 172.21.15.37:0/923178886 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fba780b7af0 0x7fba780ced00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.346+0000 7fba7cee5640 1 --2- 172.21.15.37:0/923178886 >> v2:172.21.15.92:3300/0 conn(0x7fba780cd3c0 0x7fba7815c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.346+0000 7fba7cee5640 1 -- 172.21.15.37:0/923178886 >> 172.21.15.37:0/923178886 conn(0x7fba780b3e20 msgr2=0x7fba780b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:55.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.346+0000 7fba7cee5640 1 -- 172.21.15.37:0/923178886 shutdown_connections 2024-09-17T04:32:55.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.346+0000 7fba7cee5640 1 -- 172.21.15.37:0/923178886 wait complete. 2024-09-17T04:32:55.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.348+0000 7fba7cee5640 1 Processor -- start 2024-09-17T04:32:55.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.348+0000 7fba7cee5640 1 -- start start 2024-09-17T04:32:55.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.349+0000 7fba7cee5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba780b7af0 0x7fba781547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:55.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.349+0000 7fba7cee5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba780cd3c0 0x7fba780ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:55.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.349+0000 7fba7cee5640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fba780cf240 0x7fba781f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:55.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.349+0000 7fba7cee5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fba781609c0 con 0x7fba780cd3c0 2024-09-17T04:32:55.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.349+0000 7fba7cee5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fba780b7330 con 0x7fba780b7af0 2024-09-17T04:32:55.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.349+0000 7fba7cee5640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fba78160b40 con 0x7fba780cf240 2024-09-17T04:32:55.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.350+0000 7fba75d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba780cd3c0 0x7fba780ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:55.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.350+0000 7fba75d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba780cd3c0 0x7fba780ca920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46194/0 (socket says 172.21.15.37:46194) 2024-09-17T04:32:55.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.350+0000 7fba75d9b640 1 -- 172.21.15.37:0/2337474124 learned_addr learned my addr 172.21.15.37:0/2337474124 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:32:55.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.350+0000 7fba76d9d640 1 --2- 172.21.15.37:0/2337474124 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fba780cf240 0x7fba781f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:55.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.350+0000 7fba7659c640 1 --2- 172.21.15.37:0/2337474124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba780b7af0 0x7fba781547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:55.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.350+0000 7fba75d9b640 1 -- 172.21.15.37:0/2337474124 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fba780cf240 msgr2=0x7fba781f56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.350+0000 7fba75d9b640 1 --2- 172.21.15.37:0/2337474124 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fba780cf240 0x7fba781f56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.350+0000 7fba75d9b640 1 -- 172.21.15.37:0/2337474124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba780b7af0 msgr2=0x7fba781547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.350+0000 7fba75d9b640 1 --2- 172.21.15.37:0/2337474124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba780b7af0 0x7fba781547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.350+0000 7fba75d9b640 1 -- 172.21.15.37:0/2337474124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba780c9880 con 0x7fba780cd3c0 2024-09-17T04:32:55.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.350+0000 7fba76d9d640 1 --2- 172.21.15.37:0/2337474124 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fba780cf240 0x7fba781f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:32:55.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.350+0000 7fba75d9b640 1 --2- 172.21.15.37:0/2337474124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba780cd3c0 0x7fba780ca920 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fba6802ab00 tx=0x7fba68033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:55.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.350+0000 7fba6effd640 1 -- 172.21.15.37:0/2337474124 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68005c90 con 0x7fba780cd3c0 2024-09-17T04:32:55.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.350+0000 7fba6effd640 1 -- 172.21.15.37:0/2337474124 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba68031e50 con 0x7fba780cd3c0 2024-09-17T04:32:55.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.350+0000 7fba6effd640 1 -- 172.21.15.37:0/2337474124 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68005260 con 0x7fba780cd3c0 2024-09-17T04:32:55.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.350+0000 7fba7cee5640 1 -- 172.21.15.37:0/2337474124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba780c9b60 con 0x7fba780cd3c0 2024-09-17T04:32:55.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.350+0000 7fba7cee5640 1 -- 172.21.15.37:0/2337474124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba780ca120 con 0x7fba780cd3c0 2024-09-17T04:32:55.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.351+0000 7fba6effd640 1 -- 172.21.15.37:0/2337474124 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fba680047a0 con 0x7fba780cd3c0 2024-09-17T04:32:55.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.351+0000 7fba6effd640 1 -- 172.21.15.37:0/2337474124 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fba680332d0 con 0x7fba780cd3c0 2024-09-17T04:32:55.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.351+0000 7fba7cee5640 1 -- 172.21.15.37:0/2337474124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba7806f750 con 0x7fba780cd3c0 2024-09-17T04:32:55.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.354+0000 7fba6effd640 1 -- 172.21.15.37:0/2337474124 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fba6803f030 con 0x7fba780cd3c0 2024-09-17T04:32:55.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.415+0000 7fba7cee5640 1 -- 172.21.15.37:0/2337474124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fba7815ca20 con 0x7fba780cd3c0 2024-09-17T04:32:55.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.416+0000 7fba6effd640 1 -- 172.21.15.37:0/2337474124 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v23) ==== 59+0+1472 (secure 0 0 0) 0x7fba68004c20 con 0x7fba780cd3c0 2024-09-17T04:32:55.416 INFO:teuthology.orchestra.run.smithi037.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T04:32:55.416 INFO:teuthology.orchestra.run.smithi037.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T04:32:55.416 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced client_mount_timeout 30 2024-09-17T04:32:55.416 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced debug_client 20/20 2024-09-17T04:32:55.416 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced debug_mgr 20/20 2024-09-17T04:32:55.416 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced debug_ms 1/1 2024-09-17T04:32:55.416 INFO:teuthology.orchestra.run.smithi037.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T04:32:55.416 INFO:teuthology.orchestra.run.smithi037.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T04:32:55.416 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced debug_mds 20/20 2024-09-17T04:32:55.416 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T04:32:55.416 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced debug_ms 2/2 2024-09-17T04:32:55.416 INFO:teuthology.orchestra.run.smithi037.stdout:mds dev mds_debug_frag true 2024-09-17T04:32:55.417 INFO:teuthology.orchestra.run.smithi037.stdout:mds dev mds_debug_scatterstat true 2024-09-17T04:32:55.417 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T04:32:55.417 INFO:teuthology.orchestra.run.smithi037.stdout:mds dev mds_verify_scatter true 2024-09-17T04:32:55.417 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T04:32:55.417 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T04:32:55.417 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T04:32:55.417 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced client_mount_timeout 600 2024-09-17T04:32:55.417 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced debug_client 20/20 2024-09-17T04:32:55.417 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced debug_ms 1/1 2024-09-17T04:32:55.417 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T04:32:55.417 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T04:32:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.416+0000 7fba7cee5640 1 -- 172.21.15.37:0/2337474124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba780cd3c0 msgr2=0x7fba780ca920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.416+0000 7fba7cee5640 1 --2- 172.21.15.37:0/2337474124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba780cd3c0 0x7fba780ca920 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fba6802ab00 tx=0x7fba68033c20 comp rx=0 tx=0).stop 2024-09-17T04:32:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.417+0000 7fba7cee5640 1 -- 172.21.15.37:0/2337474124 shutdown_connections 2024-09-17T04:32:55.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.417+0000 7fba7cee5640 1 --2- 172.21.15.37:0/2337474124 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fba780cf240 0x7fba781f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.417+0000 7fba7cee5640 1 --2- 172.21.15.37:0/2337474124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba780cd3c0 0x7fba780ca920 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.417+0000 7fba7cee5640 1 --2- 172.21.15.37:0/2337474124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba780b7af0 0x7fba781547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.417+0000 7fba7cee5640 1 -- 172.21.15.37:0/2337474124 >> 172.21.15.37:0/2337474124 conn(0x7fba780b3e20 msgr2=0x7fba780b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:55.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.417+0000 7fba7cee5640 1 -- 172.21.15.37:0/2337474124 shutdown_connections 2024-09-17T04:32:55.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.417+0000 7fba7cee5640 1 -- 172.21.15.37:0/2337474124 wait complete. 2024-09-17T04:32:55.427 INFO:tasks.ceph:Setting crush tunables to default 2024-09-17T04:32:55.427 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph osd crush tunables default 2024-09-17T04:32:55.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.543+0000 7f33247a1640 1 -- 172.21.15.37:0/3080851636 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f331c0cf8f0 msgr2=0x7f331c15b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.543+0000 7f33247a1640 1 --2- 172.21.15.37:0/3080851636 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f331c0cf8f0 0x7f331c15b730 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f330c009860 tx=0x7f330c02a4a0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.543+0000 7f33247a1640 1 -- 172.21.15.37:0/3080851636 shutdown_connections 2024-09-17T04:32:55.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.543+0000 7f33247a1640 1 --2- 172.21.15.37:0/3080851636 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f331c0cf8f0 0x7f331c15b730 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.544+0000 7f33247a1640 1 -- 172.21.15.37:0/3080851636 >> 172.21.15.37:0/3080851636 conn(0x7f331c069120 msgr2=0x7f331c069530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:55.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.544+0000 7f33247a1640 1 -- 172.21.15.37:0/3080851636 shutdown_connections 2024-09-17T04:32:55.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.544+0000 7f33247a1640 1 -- 172.21.15.37:0/3080851636 wait complete. 2024-09-17T04:32:55.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.545+0000 7f33247a1640 1 Processor -- start 2024-09-17T04:32:55.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.545+0000 7f33247a1640 1 -- start start 2024-09-17T04:32:55.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.547+0000 7f33247a1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f331c15bc70 0x7f331c1559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:55.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.547+0000 7f33247a1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f331c15d960 0x7f331c15f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:55.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.547+0000 7f33247a1640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f331c156170 0x7f331c15f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:55.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.547+0000 7f33247a1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f331c0cd5e0 con 0x7f331c15bc70 2024-09-17T04:32:55.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.547+0000 7f33247a1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f331c0cd460 con 0x7f331c15d960 2024-09-17T04:32:55.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.547+0000 7f33247a1640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f331c15c790 con 0x7f331c156170 2024-09-17T04:32:55.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.547+0000 7f332253d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f331c15bc70 0x7f331c1559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:55.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.547+0000 7f332253d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f331c15bc70 0x7f331c1559a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46200/0 (socket says 172.21.15.37:46200) 2024-09-17T04:32:55.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.547+0000 7f332253d640 1 -- 172.21.15.37:0/3266845104 learned_addr learned my addr 172.21.15.37:0/3266845104 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:32:55.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.547+0000 7f3322d3e640 1 --2- 172.21.15.37:0/3266845104 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f331c156170 0x7f331c15f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:55.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.547+0000 7f3321d3c640 1 --2- 172.21.15.37:0/3266845104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f331c15d960 0x7f331c15f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:55.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.547+0000 7f332253d640 1 -- 172.21.15.37:0/3266845104 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f331c156170 msgr2=0x7f331c15f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.547+0000 7f332253d640 1 --2- 172.21.15.37:0/3266845104 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f331c156170 0x7f331c15f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.547+0000 7f332253d640 1 -- 172.21.15.37:0/3266845104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f331c15d960 msgr2=0x7f331c15f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.547+0000 7f332253d640 1 --2- 172.21.15.37:0/3266845104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f331c15d960 0x7f331c15f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.547+0000 7f332253d640 1 -- 172.21.15.37:0/3266845104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f331c0ae300 con 0x7f331c15bc70 2024-09-17T04:32:55.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.547+0000 7f3321d3c640 1 --2- 172.21.15.37:0/3266845104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f331c15d960 0x7f331c15f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:32:55.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.547+0000 7f332253d640 1 --2- 172.21.15.37:0/3266845104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f331c15bc70 0x7f331c1559a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f331800c3c0 tx=0x7f331800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:55.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.548+0000 7f3312ffd640 1 -- 172.21.15.37:0/3266845104 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3318017070 con 0x7f331c15bc70 2024-09-17T04:32:55.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.548+0000 7f3312ffd640 1 -- 172.21.15.37:0/3266845104 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33180053c0 con 0x7f331c15bc70 2024-09-17T04:32:55.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.548+0000 7f33247a1640 1 -- 172.21.15.37:0/3266845104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f331c1f9420 con 0x7f331c15bc70 2024-09-17T04:32:55.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.548+0000 7f3312ffd640 1 -- 172.21.15.37:0/3266845104 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3318003850 con 0x7f331c15bc70 2024-09-17T04:32:55.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.548+0000 7f33247a1640 1 -- 172.21.15.37:0/3266845104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f331c0afbd0 con 0x7f331c15bc70 2024-09-17T04:32:55.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.548+0000 7f3312ffd640 1 -- 172.21.15.37:0/3266845104 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f331801a070 con 0x7f331c15bc70 2024-09-17T04:32:55.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.548+0000 7f3312ffd640 1 -- 172.21.15.37:0/3266845104 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f331801c780 con 0x7f331c15bc70 2024-09-17T04:32:55.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.549+0000 7f33247a1640 1 -- 172.21.15.37:0/3266845104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f331c15b730 con 0x7f331c15bc70 2024-09-17T04:32:55.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.551+0000 7f3312ffd640 1 -- 172.21.15.37:0/3266845104 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f3318021070 con 0x7f331c15bc70 2024-09-17T04:32:55.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.617+0000 7f33247a1640 1 -- 172.21.15.37:0/3266845104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f331c0b1f60 con 0x7f331c15bc70 2024-09-17T04:32:55.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.619+0000 7f3312ffd640 1 -- 172.21.15.37:0/3266845104 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f3318011d00 con 0x7f331c15bc70 2024-09-17T04:32:55.619 INFO:teuthology.orchestra.run.smithi037.stderr:adjusted tunables profile to default 2024-09-17T04:32:55.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.620+0000 7f33247a1640 1 -- 172.21.15.37:0/3266845104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f331c15bc70 msgr2=0x7f331c1559a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:55.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.620+0000 7f33247a1640 1 --2- 172.21.15.37:0/3266845104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f331c15bc70 0x7f331c1559a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f331800c3c0 tx=0x7f331800c880 comp rx=0 tx=0).stop 2024-09-17T04:32:55.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.620+0000 7f33247a1640 1 -- 172.21.15.37:0/3266845104 shutdown_connections 2024-09-17T04:32:55.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.620+0000 7f33247a1640 1 --2- 172.21.15.37:0/3266845104 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f331c156170 0x7f331c15f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.620+0000 7f33247a1640 1 --2- 172.21.15.37:0/3266845104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f331c15d960 0x7f331c15f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.620+0000 7f33247a1640 1 --2- 172.21.15.37:0/3266845104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f331c15bc70 0x7f331c1559a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:55.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.620+0000 7f33247a1640 1 -- 172.21.15.37:0/3266845104 >> 172.21.15.37:0/3266845104 conn(0x7f331c069120 msgr2=0x7f331c15c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:55.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.620+0000 7f33247a1640 1 -- 172.21.15.37:0/3266845104 shutdown_connections 2024-09-17T04:32:55.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:55.620+0000 7f33247a1640 1 -- 172.21.15.37:0/3266845104 wait complete. 2024-09-17T04:32:55.631 INFO:tasks.ceph:check_enable_crimson: False 2024-09-17T04:32:55.631 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-17T04:32:55.631 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:32:55.632 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-17T04:32:55.657 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:32:55.657 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-17T04:32:55.723 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:32:55.724 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-17T04:32:55.789 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:32:55.789 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-17T04:32:55.855 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:32:55.856 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-17T04:32:55.881 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:32:55.881 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-17T04:32:55.946 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:32:55.946 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-17T04:32:56.010 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:32:56.010 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-17T04:32:56.075 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 6ea4b8d1-0d22-4dd9-b913-9df585ef24ff 0 2024-09-17T04:32:56.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.229+0000 7fc3ff0f2640 1 -- 172.21.15.92:0/3418964949 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc3f815b620 msgr2=0x7fc3f815f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:56.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.229+0000 7fc3ff0f2640 1 --2- 172.21.15.92:0/3418964949 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc3f815b620 0x7fc3f815f300 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc3f80cfe60 tx=0x7fc3ec02a560 comp rx=0 tx=0).stop 2024-09-17T04:32:56.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.229+0000 7fc3ff0f2640 1 -- 172.21.15.92:0/3418964949 shutdown_connections 2024-09-17T04:32:56.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.229+0000 7fc3ff0f2640 1 --2- 172.21.15.92:0/3418964949 >> v2:172.21.15.92:3300/0 conn(0x7fc3f80aea30 0x7fc3f80aee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.229+0000 7fc3ff0f2640 1 --2- 172.21.15.92:0/3418964949 >> v2:172.21.15.37:3300/0 conn(0x7fc3f815f840 0x7fc3f815fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.229+0000 7fc3ff0f2640 1 --2- 172.21.15.92:0/3418964949 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc3f815b620 0x7fc3f815f300 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.229+0000 7fc3ff0f2640 1 -- 172.21.15.92:0/3418964949 >> 172.21.15.92:0/3418964949 conn(0x7fc3f80b65d0 msgr2=0x7fc3f80b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:56.229 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.229+0000 7fc3ff0f2640 1 -- 172.21.15.92:0/3418964949 shutdown_connections 2024-09-17T04:32:56.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.229+0000 7fc3ff0f2640 1 -- 172.21.15.92:0/3418964949 wait complete. 2024-09-17T04:32:56.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.230+0000 7fc3ff0f2640 1 Processor -- start 2024-09-17T04:32:56.230 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.230+0000 7fc3ff0f2640 1 -- start start 2024-09-17T04:32:56.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.232+0000 7fc3ff0f2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3f80aea30 0x7fc3f81f9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:56.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.232+0000 7fc3ff0f2640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc3f815b620 0x7fc3f814cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:56.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.232+0000 7fc3ff0f2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3f815f840 0x7fc3f80cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:56.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.232+0000 7fc3ff0f2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f80af880 con 0x7fc3f815f840 2024-09-17T04:32:56.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.232+0000 7fc3ff0f2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f80af700 con 0x7fc3f80aea30 2024-09-17T04:32:56.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.232+0000 7fc3ff0f2640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fc3f80afa00 con 0x7fc3f815b620 2024-09-17T04:32:56.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.232+0000 7fc3fce8e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3f80aea30 0x7fc3f81f9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:56.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.232+0000 7fc3f7fff640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc3f815b620 0x7fc3f814cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:56.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.232+0000 7fc3fce8e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3f80aea30 0x7fc3f81f9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34356/0 (socket says 172.21.15.92:34356) 2024-09-17T04:32:56.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.232+0000 7fc3fce8e640 1 -- 172.21.15.92:0/3924157911 learned_addr learned my addr 172.21.15.92:0/3924157911 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-17T04:32:56.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.232+0000 7fc3fd68f640 1 --2- 172.21.15.92:0/3924157911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3f815f840 0x7fc3f80cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:56.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.232+0000 7fc3fce8e640 1 -- 172.21.15.92:0/3924157911 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc3f815b620 msgr2=0x7fc3f814cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:56.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.232+0000 7fc3fce8e640 1 --2- 172.21.15.92:0/3924157911 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc3f815b620 0x7fc3f814cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.232+0000 7fc3fce8e640 1 -- 172.21.15.92:0/3924157911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3f815f840 msgr2=0x7fc3f80cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:56.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.232+0000 7fc3fce8e640 1 --2- 172.21.15.92:0/3924157911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3f815f840 0x7fc3f80cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.232+0000 7fc3fce8e640 1 -- 172.21.15.92:0/3924157911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f80ca980 con 0x7fc3f80aea30 2024-09-17T04:32:56.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.232+0000 7fc3f7fff640 1 --2- 172.21.15.92:0/3924157911 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc3f815b620 0x7fc3f814cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:32:56.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.233+0000 7fc3fce8e640 1 --2- 172.21.15.92:0/3924157911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3f80aea30 0x7fc3f81f9990 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec034040 tx=0x7fc3ec003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:56.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.233+0000 7fc3f57fa640 1 -- 172.21.15.92:0/3924157911 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec03e030 con 0x7fc3f80aea30 2024-09-17T04:32:56.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.233+0000 7fc3f57fa640 1 -- 172.21.15.92:0/3924157911 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3ec003c20 con 0x7fc3f80aea30 2024-09-17T04:32:56.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.233+0000 7fc3ff0f2640 1 -- 172.21.15.92:0/3924157911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f80cb090 con 0x7fc3f80aea30 2024-09-17T04:32:56.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.233+0000 7fc3f57fa640 1 -- 172.21.15.92:0/3924157911 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec035040 con 0x7fc3f80aea30 2024-09-17T04:32:56.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.233+0000 7fc3ff0f2640 1 -- 172.21.15.92:0/3924157911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f815dcb0 con 0x7fc3f80aea30 2024-09-17T04:32:56.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.233+0000 7fc3f57fa640 1 -- 172.21.15.92:0/3924157911 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc3ec041030 con 0x7fc3f80aea30 2024-09-17T04:32:56.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.233+0000 7fc3f57fa640 1 -- 172.21.15.92:0/3924157911 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fc3ec043af0 con 0x7fc3f80aea30 2024-09-17T04:32:56.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.234+0000 7fc3ff0f2640 1 -- 172.21.15.92:0/3924157911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3f815f300 con 0x7fc3f80aea30 2024-09-17T04:32:56.236 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.236+0000 7fc3f57fa640 1 -- 172.21.15.92:0/3924157911 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fc3ec043d40 con 0x7fc3f80aea30 2024-09-17T04:32:56.307 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.307+0000 7fc3ff0f2640 1 -- 172.21.15.92:0/3924157911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6ea4b8d1-0d22-4dd9-b913-9df585ef24ff", "id": 0} v 0) -- 0x7fc3f806f670 con 0x7fc3f80aea30 2024-09-17T04:32:56.311 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.312+0000 7fc3f57fa640 1 -- 172.21.15.92:0/3924157911 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6ea4b8d1-0d22-4dd9-b913-9df585ef24ff", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fc3ec004c70 con 0x7fc3f80aea30 2024-09-17T04:32:56.312 INFO:teuthology.orchestra.run.smithi092.stdout:0 2024-09-17T04:32:56.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.313+0000 7fc3ff0f2640 1 -- 172.21.15.92:0/3924157911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3f80aea30 msgr2=0x7fc3f81f9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:56.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.313+0000 7fc3ff0f2640 1 --2- 172.21.15.92:0/3924157911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3f80aea30 0x7fc3f81f9990 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec034040 tx=0x7fc3ec003820 comp rx=0 tx=0).stop 2024-09-17T04:32:56.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.313+0000 7fc3ff0f2640 1 -- 172.21.15.92:0/3924157911 shutdown_connections 2024-09-17T04:32:56.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.313+0000 7fc3ff0f2640 1 --2- 172.21.15.92:0/3924157911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3f815f840 0x7fc3f80cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.313+0000 7fc3ff0f2640 1 --2- 172.21.15.92:0/3924157911 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc3f815b620 0x7fc3f814cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.313+0000 7fc3ff0f2640 1 --2- 172.21.15.92:0/3924157911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3f80aea30 0x7fc3f81f9990 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.313+0000 7fc3ff0f2640 1 -- 172.21.15.92:0/3924157911 >> 172.21.15.92:0/3924157911 conn(0x7fc3f80b65d0 msgr2=0x7fc3f815acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:56.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.313+0000 7fc3ff0f2640 1 -- 172.21.15.92:0/3924157911 shutdown_connections 2024-09-17T04:32:56.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.313+0000 7fc3ff0f2640 1 -- 172.21.15.92:0/3924157911 wait complete. 2024-09-17T04:32:56.323 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new ac745a33-e8ad-4083-9c18-71bf81982b76 1 2024-09-17T04:32:56.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.437+0000 7f1928f33640 1 -- 172.21.15.92:0/1707134868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1924069080 msgr2=0x7f192415c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:56.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.437+0000 7f1928f33640 1 --2- 172.21.15.92:0/1707134868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1924069080 0x7f192415c580 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1924073100 tx=0x7f191402a560 comp rx=0 tx=0).stop 2024-09-17T04:32:56.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.437+0000 7f1928f33640 1 -- 172.21.15.92:0/1707134868 shutdown_connections 2024-09-17T04:32:56.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.437+0000 7f1928f33640 1 --2- 172.21.15.92:0/1707134868 >> v2:172.21.15.37:3300/0 conn(0x7f19240cd970 0x7f1924070420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.437+0000 7f1928f33640 1 --2- 172.21.15.92:0/1707134868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1924069080 0x7f192415c580 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.437+0000 7f1928f33640 1 -- 172.21.15.92:0/1707134868 >> 172.21.15.92:0/1707134868 conn(0x7f19240b2790 msgr2=0x7f19240b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:56.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.437+0000 7f1928f33640 1 -- 172.21.15.92:0/1707134868 shutdown_connections 2024-09-17T04:32:56.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.437+0000 7f1928f33640 1 -- 172.21.15.92:0/1707134868 wait complete. 2024-09-17T04:32:56.438 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.439+0000 7f1928f33640 1 Processor -- start 2024-09-17T04:32:56.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.439+0000 7f1928f33640 1 -- start start 2024-09-17T04:32:56.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.440+0000 7f1928f33640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1924069080 0x7f19241e6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:56.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.440+0000 7f1928f33640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f19240afbf0 0x7f19241e9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:56.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.440+0000 7f1928f33640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19240cd970 0x7f1924146270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:56.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.440+0000 7f1928f33640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1924070cf0 con 0x7f1924069080 2024-09-17T04:32:56.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.440+0000 7f1928f33640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1924070b70 con 0x7f19240cd970 2024-09-17T04:32:56.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.440+0000 7f1928f33640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f1924070e70 con 0x7f19240afbf0 2024-09-17T04:32:56.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.440+0000 7f1922d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19240cd970 0x7f1924146270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:56.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.440+0000 7f1921d9b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f19240afbf0 0x7f19241e9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:56.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.440+0000 7f1922d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19240cd970 0x7f1924146270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34362/0 (socket says 172.21.15.92:34362) 2024-09-17T04:32:56.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.440+0000 7f1922d9d640 1 -- 172.21.15.92:0/354480017 learned_addr learned my addr 172.21.15.92:0/354480017 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-17T04:32:56.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.440+0000 7f192259c640 1 --2- 172.21.15.92:0/354480017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1924069080 0x7f19241e6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:56.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.441+0000 7f1921d9b640 1 -- 172.21.15.92:0/354480017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19240cd970 msgr2=0x7f1924146270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:56.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.441+0000 7f1921d9b640 1 --2- 172.21.15.92:0/354480017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19240cd970 0x7f1924146270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.441+0000 7f1921d9b640 1 -- 172.21.15.92:0/354480017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1924069080 msgr2=0x7f19241e6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:56.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.441+0000 7f1921d9b640 1 --2- 172.21.15.92:0/354480017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1924069080 0x7f19241e6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.441+0000 7f1921d9b640 1 -- 172.21.15.92:0/354480017 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19241451d0 con 0x7f19240afbf0 2024-09-17T04:32:56.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.441+0000 7f1921d9b640 1 --2- 172.21.15.92:0/354480017 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f19240afbf0 0x7f19241e9680 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f190c00c3c0 tx=0x7f190c00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:56.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.441+0000 7f191affd640 1 -- 172.21.15.92:0/354480017 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c017070 con 0x7f19240afbf0 2024-09-17T04:32:56.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.441+0000 7f191affd640 1 -- 172.21.15.92:0/354480017 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f190c0053e0 con 0x7f19240afbf0 2024-09-17T04:32:56.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.441+0000 7f1928f33640 1 -- 172.21.15.92:0/354480017 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19241454b0 con 0x7f19240afbf0 2024-09-17T04:32:56.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.441+0000 7f191affd640 1 -- 172.21.15.92:0/354480017 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c003850 con 0x7f19240afbf0 2024-09-17T04:32:56.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.441+0000 7f1928f33640 1 -- 172.21.15.92:0/354480017 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1924145a70 con 0x7f19240afbf0 2024-09-17T04:32:56.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.441+0000 7f191affd640 1 -- 172.21.15.92:0/354480017 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f190c01a070 con 0x7f19240afbf0 2024-09-17T04:32:56.443 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.441+0000 7f191affd640 1 -- 172.21.15.92:0/354480017 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f190c011e30 con 0x7f19240afbf0 2024-09-17T04:32:56.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.442+0000 7f1928f33640 1 -- 172.21.15.92:0/354480017 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1924071390 con 0x7f19240afbf0 2024-09-17T04:32:56.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.444+0000 7f191affd640 1 -- 172.21.15.92:0/354480017 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f190c021070 con 0x7f19240afbf0 2024-09-17T04:32:56.514 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.515+0000 7f1928f33640 1 -- 172.21.15.92:0/354480017 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ac745a33-e8ad-4083-9c18-71bf81982b76", "id": 1} v 0) -- 0x7f1924070640 con 0x7f19240afbf0 2024-09-17T04:32:56.518 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.519+0000 7f191affd640 1 -- 172.21.15.92:0/354480017 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ac745a33-e8ad-4083-9c18-71bf81982b76", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f190c003ba0 con 0x7f19240afbf0 2024-09-17T04:32:56.519 INFO:teuthology.orchestra.run.smithi092.stdout:1 2024-09-17T04:32:56.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.520+0000 7f1928f33640 1 -- 172.21.15.92:0/354480017 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f19240afbf0 msgr2=0x7f19241e9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:56.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.520+0000 7f1928f33640 1 --2- 172.21.15.92:0/354480017 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f19240afbf0 0x7f19241e9680 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f190c00c3c0 tx=0x7f190c00c880 comp rx=0 tx=0).stop 2024-09-17T04:32:56.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.520+0000 7f1928f33640 1 -- 172.21.15.92:0/354480017 shutdown_connections 2024-09-17T04:32:56.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.520+0000 7f1928f33640 1 --2- 172.21.15.92:0/354480017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19240cd970 0x7f1924146270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.520+0000 7f1928f33640 1 --2- 172.21.15.92:0/354480017 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f19240afbf0 0x7f19241e9680 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.520+0000 7f1928f33640 1 --2- 172.21.15.92:0/354480017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1924069080 0x7f19241e6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.520+0000 7f1928f33640 1 -- 172.21.15.92:0/354480017 >> 172.21.15.92:0/354480017 conn(0x7f19240b2790 msgr2=0x7f19240b04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:56.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.520+0000 7f1928f33640 1 -- 172.21.15.92:0/354480017 shutdown_connections 2024-09-17T04:32:56.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.520+0000 7f1928f33640 1 -- 172.21.15.92:0/354480017 wait complete. 2024-09-17T04:32:56.531 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 3c63f3d8-132a-4735-b0bc-d64313a46503 2 2024-09-17T04:32:56.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.646+0000 7fd395edc640 1 -- 172.21.15.92:0/999987441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd39009dc60 msgr2=0x7fd3900ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:56.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.646+0000 7fd395edc640 1 --2- 172.21.15.92:0/999987441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd39009dc60 0x7fd3900ce740 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd38000a940 tx=0x7fd38002b8f0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.646+0000 7fd395edc640 1 -- 172.21.15.92:0/999987441 shutdown_connections 2024-09-17T04:32:56.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.646+0000 7fd395edc640 1 --2- 172.21.15.92:0/999987441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd39009dc60 0x7fd3900ce740 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.646+0000 7fd395edc640 1 -- 172.21.15.92:0/999987441 >> 172.21.15.92:0/999987441 conn(0x7fd3900b1ca0 msgr2=0x7fd3900b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:56.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.646+0000 7fd395edc640 1 -- 172.21.15.92:0/999987441 shutdown_connections 2024-09-17T04:32:56.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.646+0000 7fd395edc640 1 -- 172.21.15.92:0/999987441 wait complete. 2024-09-17T04:32:56.647 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.648+0000 7fd395edc640 1 Processor -- start 2024-09-17T04:32:56.648 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.648+0000 7fd395edc640 1 -- start start 2024-09-17T04:32:56.650 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.649+0000 7fd395edc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd39006ee30 0x7fd390159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:56.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.649+0000 7fd395edc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd39009dc60 0x7fd390158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:56.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.649+0000 7fd395edc640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd3900cd570 0x7fd39015a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:56.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.649+0000 7fd395edc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd39006f5d0 con 0x7fd39009dc60 2024-09-17T04:32:56.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.649+0000 7fd395edc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd39006f450 con 0x7fd39006ee30 2024-09-17T04:32:56.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.649+0000 7fd395edc640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fd39006f750 con 0x7fd3900cd570 2024-09-17T04:32:56.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.649+0000 7fd38f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd39006ee30 0x7fd390159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:56.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.649+0000 7fd38ffff640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd3900cd570 0x7fd39015a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:56.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.649+0000 7fd38f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd39006ee30 0x7fd390159cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34374/0 (socket says 172.21.15.92:34374) 2024-09-17T04:32:56.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.649+0000 7fd38f7fe640 1 -- 172.21.15.92:0/30408044 learned_addr learned my addr 172.21.15.92:0/30408044 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-17T04:32:56.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.649+0000 7fd387fff640 1 --2- 172.21.15.92:0/30408044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd39009dc60 0x7fd390158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:56.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.650+0000 7fd38ffff640 1 -- 172.21.15.92:0/30408044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd39006ee30 msgr2=0x7fd390159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:56.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.650+0000 7fd38ffff640 1 --2- 172.21.15.92:0/30408044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd39006ee30 0x7fd390159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.650+0000 7fd38ffff640 1 -- 172.21.15.92:0/30408044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd39009dc60 msgr2=0x7fd390158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:56.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.650+0000 7fd38ffff640 1 --2- 172.21.15.92:0/30408044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd39009dc60 0x7fd390158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.650+0000 7fd38ffff640 1 -- 172.21.15.92:0/30408044 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39014b310 con 0x7fd3900cd570 2024-09-17T04:32:56.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.650+0000 7fd387fff640 1 --2- 172.21.15.92:0/30408044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd39009dc60 0x7fd390158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:32:56.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.650+0000 7fd38ffff640 1 --2- 172.21.15.92:0/30408044 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd3900cd570 0x7fd39015a9c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd38800c700 tx=0x7fd38800cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:56.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.650+0000 7fd38cff9640 1 -- 172.21.15.92:0/30408044 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd388003b70 con 0x7fd3900cd570 2024-09-17T04:32:56.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.650+0000 7fd38cff9640 1 -- 172.21.15.92:0/30408044 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd388003d10 con 0x7fd3900cd570 2024-09-17T04:32:56.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.650+0000 7fd395edc640 1 -- 172.21.15.92:0/30408044 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd39014b5f0 con 0x7fd3900cd570 2024-09-17T04:32:56.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.650+0000 7fd38cff9640 1 -- 172.21.15.92:0/30408044 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd388005330 con 0x7fd3900cd570 2024-09-17T04:32:56.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.650+0000 7fd395edc640 1 -- 172.21.15.92:0/30408044 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd39014bbb0 con 0x7fd3900cd570 2024-09-17T04:32:56.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.650+0000 7fd38cff9640 1 -- 172.21.15.92:0/30408044 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd3880054d0 con 0x7fd3900cd570 2024-09-17T04:32:56.652 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.650+0000 7fd38cff9640 1 -- 172.21.15.92:0/30408044 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fd388012890 con 0x7fd3900cd570 2024-09-17T04:32:56.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.651+0000 7fd395edc640 1 -- 172.21.15.92:0/30408044 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3901f4530 con 0x7fd3900cd570 2024-09-17T04:32:56.653 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.653+0000 7fd38cff9640 1 -- 172.21.15.92:0/30408044 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd388012470 con 0x7fd3900cd570 2024-09-17T04:32:56.724 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.724+0000 7fd395edc640 1 -- 172.21.15.92:0/30408044 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3c63f3d8-132a-4735-b0bc-d64313a46503", "id": 2} v 0) -- 0x7fd3900ce740 con 0x7fd3900cd570 2024-09-17T04:32:56.728 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.728+0000 7fd38cff9640 1 -- 172.21.15.92:0/30408044 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3c63f3d8-132a-4735-b0bc-d64313a46503", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fd388012b70 con 0x7fd3900cd570 2024-09-17T04:32:56.728 INFO:teuthology.orchestra.run.smithi092.stdout:2 2024-09-17T04:32:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.729+0000 7fd395edc640 1 -- 172.21.15.92:0/30408044 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd3900cd570 msgr2=0x7fd39015a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.729+0000 7fd395edc640 1 --2- 172.21.15.92:0/30408044 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd3900cd570 0x7fd39015a9c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd38800c700 tx=0x7fd38800cbc0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.729+0000 7fd395edc640 1 -- 172.21.15.92:0/30408044 shutdown_connections 2024-09-17T04:32:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.729+0000 7fd395edc640 1 --2- 172.21.15.92:0/30408044 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd3900cd570 0x7fd39015a9c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.729+0000 7fd395edc640 1 --2- 172.21.15.92:0/30408044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd39009dc60 0x7fd390158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.729+0000 7fd395edc640 1 --2- 172.21.15.92:0/30408044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd39006ee30 0x7fd390159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.729+0000 7fd395edc640 1 -- 172.21.15.92:0/30408044 >> 172.21.15.92:0/30408044 conn(0x7fd3900b1ca0 msgr2=0x7fd390072db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.729+0000 7fd395edc640 1 -- 172.21.15.92:0/30408044 shutdown_connections 2024-09-17T04:32:56.729 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.729+0000 7fd395edc640 1 -- 172.21.15.92:0/30408044 wait complete. 2024-09-17T04:32:56.739 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 4f8c6ca2-ea1d-4610-9127-52c4dd8bc8e3 3 2024-09-17T04:32:56.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.851+0000 7f972cedc640 1 -- 172.21.15.92:0/2200144641 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f97280b2560 msgr2=0x7f97280d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:56.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.851+0000 7f972cedc640 1 --2- 172.21.15.92:0/2200144641 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f97280b2560 0x7f97280d1e20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9718005fd0 tx=0x7f971802a540 comp rx=0 tx=0).stop 2024-09-17T04:32:56.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.851+0000 7f972cedc640 1 -- 172.21.15.92:0/2200144641 shutdown_connections 2024-09-17T04:32:56.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.851+0000 7f972cedc640 1 --2- 172.21.15.92:0/2200144641 >> v2:172.21.15.37:3300/0 conn(0x7f97280cfed0 0x7f97280b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.851+0000 7f972cedc640 1 --2- 172.21.15.92:0/2200144641 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f97280b2560 0x7f97280d1e20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.851+0000 7f972cedc640 1 -- 172.21.15.92:0/2200144641 >> 172.21.15.92:0/2200144641 conn(0x7f97280d1300 msgr2=0x7f97280d1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:56.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.851+0000 7f972cedc640 1 -- 172.21.15.92:0/2200144641 shutdown_connections 2024-09-17T04:32:56.852 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.851+0000 7f972cedc640 1 -- 172.21.15.92:0/2200144641 wait complete. 2024-09-17T04:32:56.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.853+0000 7f972cedc640 1 Processor -- start 2024-09-17T04:32:56.853 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.853+0000 7f972cedc640 1 -- start start 2024-09-17T04:32:56.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.855+0000 7f972cedc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97280b2560 0x7f97281f4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:56.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.855+0000 7f972cedc640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f97280ce210 0x7f97281e0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:56.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.855+0000 7f972cedc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97280cfed0 0x7f9728147e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:56.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.855+0000 7f972cedc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f97280b5750 con 0x7f97280b2560 2024-09-17T04:32:56.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.855+0000 7f972cedc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f97280b55d0 con 0x7f97280cfed0 2024-09-17T04:32:56.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.855+0000 7f972cedc640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f97280b58d0 con 0x7f97280ce210 2024-09-17T04:32:56.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.855+0000 7f9726d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97280cfed0 0x7f9728147e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:56.856 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.855+0000 7f972659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97280b2560 0x7f97281f4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:56.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.855+0000 7f9725d9b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f97280ce210 0x7f97281e0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:56.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.855+0000 7f972659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97280b2560 0x7f97281f4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.92:58566/0 (socket says 172.21.15.92:58566) 2024-09-17T04:32:56.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.855+0000 7f972659c640 1 -- 172.21.15.92:0/1678810793 learned_addr learned my addr 172.21.15.92:0/1678810793 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-17T04:32:56.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.855+0000 7f9726d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97280cfed0 0x7f9728147e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34378/0 (socket says 172.21.15.92:34378) 2024-09-17T04:32:56.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.855+0000 7f972659c640 1 -- 172.21.15.92:0/1678810793 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f97280ce210 msgr2=0x7f97281e0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:56.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.855+0000 7f972659c640 1 --2- 172.21.15.92:0/1678810793 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f97280ce210 0x7f97281e0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.855+0000 7f972659c640 1 -- 172.21.15.92:0/1678810793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97280cfed0 msgr2=0x7f9728147e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:56.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.855+0000 7f972659c640 1 --2- 172.21.15.92:0/1678810793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97280cfed0 0x7f9728147e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.855+0000 7f972659c640 1 -- 172.21.15.92:0/1678810793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f972814aaa0 con 0x7f97280b2560 2024-09-17T04:32:56.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.855+0000 7f9725d9b640 1 --2- 172.21.15.92:0/1678810793 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f97280ce210 0x7f97281e0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:32:56.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.855+0000 7f972659c640 1 --2- 172.21.15.92:0/1678810793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97280b2560 0x7f97281f4280 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9718034040 tx=0x7f97180038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:56.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.856+0000 7f9716ffd640 1 -- 172.21.15.92:0/1678810793 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971803e030 con 0x7f97280b2560 2024-09-17T04:32:56.857 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.856+0000 7f9716ffd640 1 -- 172.21.15.92:0/1678810793 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9718003db0 con 0x7f97280b2560 2024-09-17T04:32:56.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.856+0000 7f972cedc640 1 -- 172.21.15.92:0/1678810793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f972814ad80 con 0x7f97280b2560 2024-09-17T04:32:56.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.856+0000 7f9716ffd640 1 -- 172.21.15.92:0/1678810793 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9718035040 con 0x7f97280b2560 2024-09-17T04:32:56.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.856+0000 7f972cedc640 1 -- 172.21.15.92:0/1678810793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f972814b340 con 0x7f97280b2560 2024-09-17T04:32:56.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.856+0000 7f9716ffd640 1 -- 172.21.15.92:0/1678810793 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9718041030 con 0x7f97280b2560 2024-09-17T04:32:56.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.856+0000 7f9716ffd640 1 -- 172.21.15.92:0/1678810793 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f9718043cb0 con 0x7f97280b2560 2024-09-17T04:32:56.858 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.857+0000 7f972cedc640 1 -- 172.21.15.92:0/1678810793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97280d1e20 con 0x7f97280b2560 2024-09-17T04:32:56.859 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.859+0000 7f9716ffd640 1 -- 172.21.15.92:0/1678810793 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f9718043410 con 0x7f97280b2560 2024-09-17T04:32:56.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.929+0000 7f972cedc640 1 -- 172.21.15.92:0/1678810793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4f8c6ca2-ea1d-4610-9127-52c4dd8bc8e3", "id": 3} v 0) -- 0x7f97280b0030 con 0x7f97280b2560 2024-09-17T04:32:56.931 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.931+0000 7f9716ffd640 1 -- 172.21.15.92:0/1678810793 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4f8c6ca2-ea1d-4610-9127-52c4dd8bc8e3", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f9718004ed0 con 0x7f97280b2560 2024-09-17T04:32:56.931 INFO:teuthology.orchestra.run.smithi092.stdout:3 2024-09-17T04:32:56.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.933+0000 7f972cedc640 1 -- 172.21.15.92:0/1678810793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97280b2560 msgr2=0x7f97281f4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:56.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.933+0000 7f972cedc640 1 --2- 172.21.15.92:0/1678810793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97280b2560 0x7f97281f4280 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9718034040 tx=0x7f97180038a0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.933+0000 7f972cedc640 1 -- 172.21.15.92:0/1678810793 shutdown_connections 2024-09-17T04:32:56.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.933+0000 7f972cedc640 1 --2- 172.21.15.92:0/1678810793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97280cfed0 0x7f9728147e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.933+0000 7f972cedc640 1 --2- 172.21.15.92:0/1678810793 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f97280ce210 0x7f97281e0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.933+0000 7f972cedc640 1 --2- 172.21.15.92:0/1678810793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97280b2560 0x7f97281f4280 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:56.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.933+0000 7f972cedc640 1 -- 172.21.15.92:0/1678810793 >> 172.21.15.92:0/1678810793 conn(0x7f97280d1300 msgr2=0x7f97280b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:56.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.933+0000 7f972cedc640 1 -- 172.21.15.92:0/1678810793 shutdown_connections 2024-09-17T04:32:56.933 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:56.933+0000 7f972cedc640 1 -- 172.21.15.92:0/1678810793 wait complete. 2024-09-17T04:32:56.943 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 63d0c4ab-f115-47b4-9611-12a10153473a 4 2024-09-17T04:32:57.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.056+0000 7fafb719b640 1 -- 172.21.15.92:0/2859013988 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fafb015c830 msgr2=0x7fafb00d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:57.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.056+0000 7fafb719b640 1 --2- 172.21.15.92:0/2859013988 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fafb015c830 0x7fafb00d0240 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fafa4005fd0 tx=0x7fafa402a5f0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.056+0000 7fafb719b640 1 -- 172.21.15.92:0/2859013988 shutdown_connections 2024-09-17T04:32:57.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.056+0000 7fafb719b640 1 --2- 172.21.15.92:0/2859013988 >> v2:172.21.15.92:3300/0 conn(0x7fafb00cd260 0x7fafb00b5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.056+0000 7fafb719b640 1 --2- 172.21.15.92:0/2859013988 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fafb015c830 0x7fafb00d0240 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.056+0000 7fafb719b640 1 -- 172.21.15.92:0/2859013988 >> 172.21.15.92:0/2859013988 conn(0x7fafb0071050 msgr2=0x7fafb0071460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:57.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.056+0000 7fafb719b640 1 -- 172.21.15.92:0/2859013988 shutdown_connections 2024-09-17T04:32:57.057 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.056+0000 7fafb719b640 1 -- 172.21.15.92:0/2859013988 wait complete. 2024-09-17T04:32:57.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.058+0000 7fafb719b640 1 Processor -- start 2024-09-17T04:32:57.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.058+0000 7fafb719b640 1 -- start start 2024-09-17T04:32:57.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.059+0000 7fafb719b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafb00d1a10 0x7fafb01f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:57.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.059+0000 7fafb719b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafb015c830 0x7fafb00b75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:57.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.059+0000 7fafb719b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fafb006fa60 0x7fafb00b7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:57.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.059+0000 7fafb719b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fafb00b62e0 con 0x7fafb00d1a10 2024-09-17T04:32:57.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.059+0000 7fafb719b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fafb00b6160 con 0x7fafb015c830 2024-09-17T04:32:57.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.059+0000 7fafb719b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fafb00b6460 con 0x7fafb006fa60 2024-09-17T04:32:57.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.060+0000 7fafb5738640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fafb006fa60 0x7fafb00b7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:57.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.060+0000 7fafb5738640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fafb006fa60 0x7fafb00b7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.92:38126/0 (socket says 172.21.15.92:38126) 2024-09-17T04:32:57.061 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.060+0000 7fafb5738640 1 -- 172.21.15.92:0/635721505 learned_addr learned my addr 172.21.15.92:0/635721505 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-17T04:32:57.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.060+0000 7fafaffff640 1 --2- 172.21.15.92:0/635721505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafb015c830 0x7fafb00b75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:57.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.060+0000 7fafb4f37640 1 --2- 172.21.15.92:0/635721505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafb00d1a10 0x7fafb01f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:57.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.060+0000 7fafb5738640 1 -- 172.21.15.92:0/635721505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafb015c830 msgr2=0x7fafb00b75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:57.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.060+0000 7fafb5738640 1 --2- 172.21.15.92:0/635721505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafb015c830 0x7fafb00b75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.060+0000 7fafb5738640 1 -- 172.21.15.92:0/635721505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafb00d1a10 msgr2=0x7fafb01f4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:57.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.060+0000 7fafb5738640 1 --2- 172.21.15.92:0/635721505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafb00d1a10 0x7fafb01f4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.060+0000 7fafb5738640 1 -- 172.21.15.92:0/635721505 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb01f5230 con 0x7fafb006fa60 2024-09-17T04:32:57.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.060+0000 7fafb5738640 1 --2- 172.21.15.92:0/635721505 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fafb006fa60 0x7fafb00b7b10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fafa800ebf0 tx=0x7fafa800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:57.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.060+0000 7fafad7fa640 1 -- 172.21.15.92:0/635721505 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa8003b70 con 0x7fafb006fa60 2024-09-17T04:32:57.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.060+0000 7fafad7fa640 1 -- 172.21.15.92:0/635721505 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafa8003d10 con 0x7fafb006fa60 2024-09-17T04:32:57.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.060+0000 7fafad7fa640 1 -- 172.21.15.92:0/635721505 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa8004280 con 0x7fafb006fa60 2024-09-17T04:32:57.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.060+0000 7fafb719b640 1 -- 172.21.15.92:0/635721505 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb01f8ed0 con 0x7fafb006fa60 2024-09-17T04:32:57.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.060+0000 7fafb719b640 1 -- 172.21.15.92:0/635721505 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb01f9490 con 0x7fafb006fa60 2024-09-17T04:32:57.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.061+0000 7fafad7fa640 1 -- 172.21.15.92:0/635721505 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fafa8004420 con 0x7fafb006fa60 2024-09-17T04:32:57.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.061+0000 7fafad7fa640 1 -- 172.21.15.92:0/635721505 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fafa8021430 con 0x7fafb006fa60 2024-09-17T04:32:57.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.061+0000 7fafb719b640 1 -- 172.21.15.92:0/635721505 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb01f9870 con 0x7fafb006fa60 2024-09-17T04:32:57.063 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.063+0000 7fafad7fa640 1 -- 172.21.15.92:0/635721505 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fafb01f9870 con 0x7fafb006fa60 2024-09-17T04:32:57.134 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.134+0000 7fafb719b640 1 -- 172.21.15.92:0/635721505 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd new", "uuid": "63d0c4ab-f115-47b4-9611-12a10153473a", "id": 4} v 0) -- 0x7fafb00b6970 con 0x7fafb006fa60 2024-09-17T04:32:57.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.137+0000 7fafad7fa640 1 -- 172.21.15.92:0/635721505 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "63d0c4ab-f115-47b4-9611-12a10153473a", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fafa8005020 con 0x7fafb006fa60 2024-09-17T04:32:57.137 INFO:teuthology.orchestra.run.smithi092.stdout:4 2024-09-17T04:32:57.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.138+0000 7fafb719b640 1 -- 172.21.15.92:0/635721505 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fafb006fa60 msgr2=0x7fafb00b7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:57.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.138+0000 7fafb719b640 1 --2- 172.21.15.92:0/635721505 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fafb006fa60 0x7fafb00b7b10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fafa800ebf0 tx=0x7fafa800c6a0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.138+0000 7fafb719b640 1 -- 172.21.15.92:0/635721505 shutdown_connections 2024-09-17T04:32:57.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.138+0000 7fafb719b640 1 --2- 172.21.15.92:0/635721505 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fafb006fa60 0x7fafb00b7b10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.138+0000 7fafb719b640 1 --2- 172.21.15.92:0/635721505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafb015c830 0x7fafb00b75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.138+0000 7fafb719b640 1 --2- 172.21.15.92:0/635721505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafb00d1a10 0x7fafb01f4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.138+0000 7fafb719b640 1 -- 172.21.15.92:0/635721505 >> 172.21.15.92:0/635721505 conn(0x7fafb0071050 msgr2=0x7fafb00b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:57.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.138+0000 7fafb719b640 1 -- 172.21.15.92:0/635721505 shutdown_connections 2024-09-17T04:32:57.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.138+0000 7fafb719b640 1 -- 172.21.15.92:0/635721505 wait complete. 2024-09-17T04:32:57.149 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 4c56c04d-0c07-4f76-9d75-6335121310ff 5 2024-09-17T04:32:57.262 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.262+0000 7f8021815640 1 -- 172.21.15.92:0/1391781080 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f801c0ccf80 msgr2=0x7f801c0d07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:57.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.262+0000 7f8021815640 1 --2- 172.21.15.92:0/1391781080 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f801c0ccf80 0x7f801c0d07c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8014005fd0 tx=0x7f801402aa10 comp rx=0 tx=0).stop 2024-09-17T04:32:57.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.262+0000 7f8021815640 1 -- 172.21.15.92:0/1391781080 shutdown_connections 2024-09-17T04:32:57.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.262+0000 7f8021815640 1 --2- 172.21.15.92:0/1391781080 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f801c0ccf80 0x7f801c0d07c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.262+0000 7f8021815640 1 --2- 172.21.15.92:0/1391781080 >> v2:172.21.15.37:3300/0 conn(0x7f801c160bf0 0x7f801c15c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.262+0000 7f8021815640 1 -- 172.21.15.92:0/1391781080 >> 172.21.15.92:0/1391781080 conn(0x7f801c0714d0 msgr2=0x7f801c071910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:57.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.262+0000 7f8021815640 1 -- 172.21.15.92:0/1391781080 shutdown_connections 2024-09-17T04:32:57.263 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.262+0000 7f8021815640 1 -- 172.21.15.92:0/1391781080 wait complete. 2024-09-17T04:32:57.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.264+0000 7f8021815640 1 Processor -- start 2024-09-17T04:32:57.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.264+0000 7f8021815640 1 -- start start 2024-09-17T04:32:57.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.265+0000 7f8021815640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f801c0b5070 0x7f801c150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:57.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.265+0000 7f8021815640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f801c160bf0 0x7f801c0c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:57.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.265+0000 7f8021815640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f801c1515f0 0x7f801c0c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:57.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.265+0000 7f8021815640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f801c0b5600 con 0x7f801c160bf0 2024-09-17T04:32:57.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.265+0000 7f8021815640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f801c0b5480 con 0x7f801c0b5070 2024-09-17T04:32:57.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.265+0000 7f8021815640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f801c0d0d00 con 0x7f801c1515f0 2024-09-17T04:32:57.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.266+0000 7f801b7fe640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f801c1515f0 0x7f801c0c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:57.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.266+0000 7f801affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f801c0b5070 0x7f801c150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:57.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.266+0000 7f801b7fe640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f801c1515f0 0x7f801c0c80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.92:38156/0 (socket says 172.21.15.92:38156) 2024-09-17T04:32:57.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.266+0000 7f801b7fe640 1 -- 172.21.15.92:0/2290705298 learned_addr learned my addr 172.21.15.92:0/2290705298 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-17T04:32:57.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.266+0000 7f801a7fc640 1 --2- 172.21.15.92:0/2290705298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f801c160bf0 0x7f801c0c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:57.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.266+0000 7f801b7fe640 1 -- 172.21.15.92:0/2290705298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f801c0b5070 msgr2=0x7f801c150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:57.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.266+0000 7f801b7fe640 1 --2- 172.21.15.92:0/2290705298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f801c0b5070 0x7f801c150dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.266+0000 7f801b7fe640 1 -- 172.21.15.92:0/2290705298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f801c160bf0 msgr2=0x7f801c0c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:57.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.266+0000 7f801b7fe640 1 --2- 172.21.15.92:0/2290705298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f801c160bf0 0x7f801c0c7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.266+0000 7f801b7fe640 1 -- 172.21.15.92:0/2290705298 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801c15ff10 con 0x7f801c1515f0 2024-09-17T04:32:57.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.266+0000 7f801a7fc640 1 --2- 172.21.15.92:0/2290705298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f801c160bf0 0x7f801c0c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:32:57.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.266+0000 7f801b7fe640 1 --2- 172.21.15.92:0/2290705298 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f801c1515f0 0x7f801c0c80b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8014009d90 tx=0x7f8014003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:57.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.266+0000 7f8003fff640 1 -- 172.21.15.92:0/2290705298 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8014033050 con 0x7f801c1515f0 2024-09-17T04:32:57.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.266+0000 7f8003fff640 1 -- 172.21.15.92:0/2290705298 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8014004450 con 0x7f801c1515f0 2024-09-17T04:32:57.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.266+0000 7f8021815640 1 -- 172.21.15.92:0/2290705298 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f801c1f6ec0 con 0x7f801c1515f0 2024-09-17T04:32:57.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.266+0000 7f8003fff640 1 -- 172.21.15.92:0/2290705298 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8014004860 con 0x7f801c1515f0 2024-09-17T04:32:57.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.266+0000 7f8021815640 1 -- 172.21.15.92:0/2290705298 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f801c1f7480 con 0x7f801c1515f0 2024-09-17T04:32:57.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.267+0000 7f8003fff640 1 -- 172.21.15.92:0/2290705298 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8014036070 con 0x7f801c1515f0 2024-09-17T04:32:57.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.267+0000 7f8003fff640 1 -- 172.21.15.92:0/2290705298 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f801404b9c0 con 0x7f801c1515f0 2024-09-17T04:32:57.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.267+0000 7f8021815640 1 -- 172.21.15.92:0/2290705298 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f801c0d11b0 con 0x7f801c1515f0 2024-09-17T04:32:57.271 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.271+0000 7f8003fff640 1 -- 172.21.15.92:0/2290705298 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f8014062860 con 0x7f801c1515f0 2024-09-17T04:32:57.341 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.342+0000 7f8021815640 1 -- 172.21.15.92:0/2290705298 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4c56c04d-0c07-4f76-9d75-6335121310ff", "id": 5} v 0) -- 0x7f801c0b1a60 con 0x7f801c1515f0 2024-09-17T04:32:57.346 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.346+0000 7f8003fff640 1 -- 172.21.15.92:0/2290705298 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4c56c04d-0c07-4f76-9d75-6335121310ff", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f8014050030 con 0x7f801c1515f0 2024-09-17T04:32:57.346 INFO:teuthology.orchestra.run.smithi092.stdout:5 2024-09-17T04:32:57.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.347+0000 7f8021815640 1 -- 172.21.15.92:0/2290705298 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f801c1515f0 msgr2=0x7f801c0c80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:57.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.347+0000 7f8021815640 1 --2- 172.21.15.92:0/2290705298 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f801c1515f0 0x7f801c0c80b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8014009d90 tx=0x7f8014003f10 comp rx=0 tx=0).stop 2024-09-17T04:32:57.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.347+0000 7f8021815640 1 -- 172.21.15.92:0/2290705298 shutdown_connections 2024-09-17T04:32:57.347 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.347+0000 7f8021815640 1 --2- 172.21.15.92:0/2290705298 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f801c1515f0 0x7f801c0c80b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.347+0000 7f8021815640 1 --2- 172.21.15.92:0/2290705298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f801c160bf0 0x7f801c0c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.347+0000 7f8021815640 1 --2- 172.21.15.92:0/2290705298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f801c0b5070 0x7f801c150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.347+0000 7f8021815640 1 -- 172.21.15.92:0/2290705298 >> 172.21.15.92:0/2290705298 conn(0x7f801c0714d0 msgr2=0x7f801c0718e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:57.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.347+0000 7f8021815640 1 -- 172.21.15.92:0/2290705298 shutdown_connections 2024-09-17T04:32:57.348 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.347+0000 7f8021815640 1 -- 172.21.15.92:0/2290705298 wait complete. 2024-09-17T04:32:57.357 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new b30eae7c-096a-4d24-ac51-96b8b13ecaee 6 2024-09-17T04:32:57.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.469+0000 7f6efa3a8640 1 -- 172.21.15.92:0/2756264714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef40ce890 msgr2=0x7f6ef4166690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:57.469 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.469+0000 7f6efa3a8640 1 --2- 172.21.15.92:0/2756264714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef40ce890 0x7f6ef4166690 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6eec005fd0 tx=0x7f6eec02a940 comp rx=0 tx=0).stop 2024-09-17T04:32:57.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.469+0000 7f6efa3a8640 1 -- 172.21.15.92:0/2756264714 shutdown_connections 2024-09-17T04:32:57.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.469+0000 7f6efa3a8640 1 --2- 172.21.15.92:0/2756264714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef40ce890 0x7f6ef4166690 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.469+0000 7f6efa3a8640 1 -- 172.21.15.92:0/2756264714 >> 172.21.15.92:0/2756264714 conn(0x7f6ef40b65d0 msgr2=0x7f6ef40b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:57.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.469+0000 7f6efa3a8640 1 -- 172.21.15.92:0/2756264714 shutdown_connections 2024-09-17T04:32:57.470 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.469+0000 7f6efa3a8640 1 -- 172.21.15.92:0/2756264714 wait complete. 2024-09-17T04:32:57.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.471+0000 7f6efa3a8640 1 Processor -- start 2024-09-17T04:32:57.471 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.471+0000 7f6efa3a8640 1 -- start start 2024-09-17T04:32:57.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.472+0000 7f6efa3a8640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6ef40ce890 0x7f6ef415eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:57.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.473+0000 7f6efa3a8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef415b620 0x7f6ef4144540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:57.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.473+0000 7f6efa3a8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ef41569b0 0x7f6ef41531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:57.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.473+0000 7f6efa3a8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef4166f60 con 0x7f6ef415b620 2024-09-17T04:32:57.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.473+0000 7f6efa3a8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef4166de0 con 0x7f6ef41569b0 2024-09-17T04:32:57.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.473+0000 7f6efa3a8640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f6ef41670e0 con 0x7f6ef40ce890 2024-09-17T04:32:57.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.473+0000 7f6ef3fff640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6ef40ce890 0x7f6ef415eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:57.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.473+0000 7f6ef8945640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ef41569b0 0x7f6ef41531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:57.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.473+0000 7f6ef8945640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ef41569b0 0x7f6ef41531b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34426/0 (socket says 172.21.15.92:34426) 2024-09-17T04:32:57.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.473+0000 7f6ef3fff640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6ef40ce890 0x7f6ef415eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.92:38158/0 (socket says 172.21.15.92:38158) 2024-09-17T04:32:57.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.473+0000 7f6ef37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef415b620 0x7f6ef4144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:57.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.473+0000 7f6ef8945640 1 -- 172.21.15.92:0/2072563463 learned_addr learned my addr 172.21.15.92:0/2072563463 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-17T04:32:57.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.473+0000 7f6ef3fff640 1 -- 172.21.15.92:0/2072563463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ef41569b0 msgr2=0x7f6ef41531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:57.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.473+0000 7f6ef3fff640 1 --2- 172.21.15.92:0/2072563463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ef41569b0 0x7f6ef41531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.473+0000 7f6ef3fff640 1 -- 172.21.15.92:0/2072563463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef415b620 msgr2=0x7f6ef4144540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:57.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.473+0000 7f6ef3fff640 1 --2- 172.21.15.92:0/2072563463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef415b620 0x7f6ef4144540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.473+0000 7f6ef3fff640 1 -- 172.21.15.92:0/2072563463 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef4152430 con 0x7f6ef40ce890 2024-09-17T04:32:57.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.473+0000 7f6ef8945640 1 --2- 172.21.15.92:0/2072563463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ef41569b0 0x7f6ef41531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:32:57.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.473+0000 7f6ef37fe640 1 --2- 172.21.15.92:0/2072563463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef415b620 0x7f6ef4144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:32:57.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.473+0000 7f6ef3fff640 1 --2- 172.21.15.92:0/2072563463 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6ef40ce890 0x7f6ef415eaf0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6edc001480 tx=0x7f6edc001880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:57.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.473+0000 7f6ef0ff9640 1 -- 172.21.15.92:0/2072563463 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc0052d0 con 0x7f6ef40ce890 2024-09-17T04:32:57.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.473+0000 7f6ef0ff9640 1 -- 172.21.15.92:0/2072563463 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6edc00f690 con 0x7f6ef40ce890 2024-09-17T04:32:57.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.473+0000 7f6efa3a8640 1 -- 172.21.15.92:0/2072563463 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef41526d0 con 0x7f6ef40ce890 2024-09-17T04:32:57.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.473+0000 7f6ef0ff9640 1 -- 172.21.15.92:0/2072563463 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc00fae0 con 0x7f6ef40ce890 2024-09-17T04:32:57.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.474+0000 7f6efa3a8640 1 -- 172.21.15.92:0/2072563463 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef4152c90 con 0x7f6ef40ce890 2024-09-17T04:32:57.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.474+0000 7f6ef0ff9640 1 -- 172.21.15.92:0/2072563463 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6edc003980 con 0x7f6ef40ce890 2024-09-17T04:32:57.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.474+0000 7f6ef0ff9640 1 -- 172.21.15.92:0/2072563463 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f6edc010ad0 con 0x7f6ef40ce890 2024-09-17T04:32:57.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.475+0000 7f6efa3a8640 1 -- 172.21.15.92:0/2072563463 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ef415fac0 con 0x7f6ef40ce890 2024-09-17T04:32:57.477 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.477+0000 7f6ef0ff9640 1 -- 172.21.15.92:0/2072563463 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f6edc010d70 con 0x7f6ef40ce890 2024-09-17T04:32:57.547 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.547+0000 7f6efa3a8640 1 -- 172.21.15.92:0/2072563463 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b30eae7c-096a-4d24-ac51-96b8b13ecaee", "id": 6} v 0) -- 0x7f6ef406f670 con 0x7f6ef40ce890 2024-09-17T04:32:57.551 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.551+0000 7f6ef0ff9640 1 -- 172.21.15.92:0/2072563463 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b30eae7c-096a-4d24-ac51-96b8b13ecaee", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f6edc00fc80 con 0x7f6ef40ce890 2024-09-17T04:32:57.551 INFO:teuthology.orchestra.run.smithi092.stdout:6 2024-09-17T04:32:57.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.552+0000 7f6efa3a8640 1 -- 172.21.15.92:0/2072563463 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6ef40ce890 msgr2=0x7f6ef415eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:57.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.552+0000 7f6efa3a8640 1 --2- 172.21.15.92:0/2072563463 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6ef40ce890 0x7f6ef415eaf0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6edc001480 tx=0x7f6edc001880 comp rx=0 tx=0).stop 2024-09-17T04:32:57.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.552+0000 7f6efa3a8640 1 -- 172.21.15.92:0/2072563463 shutdown_connections 2024-09-17T04:32:57.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.552+0000 7f6efa3a8640 1 --2- 172.21.15.92:0/2072563463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ef41569b0 0x7f6ef41531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.552+0000 7f6efa3a8640 1 --2- 172.21.15.92:0/2072563463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ef415b620 0x7f6ef4144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.552+0000 7f6efa3a8640 1 --2- 172.21.15.92:0/2072563463 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6ef40ce890 0x7f6ef415eaf0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.552+0000 7f6efa3a8640 1 -- 172.21.15.92:0/2072563463 >> 172.21.15.92:0/2072563463 conn(0x7f6ef40b65d0 msgr2=0x7f6ef4070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:57.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.552+0000 7f6efa3a8640 1 -- 172.21.15.92:0/2072563463 shutdown_connections 2024-09-17T04:32:57.552 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.552+0000 7f6efa3a8640 1 -- 172.21.15.92:0/2072563463 wait complete. 2024-09-17T04:32:57.562 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 62f12540-d374-4a4c-9ff7-63b7304fe77f 7 2024-09-17T04:32:57.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.674+0000 7f262759b640 1 -- 172.21.15.92:0/1192268498 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f26200ce890 msgr2=0x7f2620166690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:57.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.674+0000 7f262759b640 1 --2- 172.21.15.92:0/1192268498 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f26200ce890 0x7f2620166690 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2618009ca0 tx=0x7f261802ab10 comp rx=0 tx=0).stop 2024-09-17T04:32:57.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.674+0000 7f262759b640 1 -- 172.21.15.92:0/1192268498 shutdown_connections 2024-09-17T04:32:57.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.674+0000 7f262759b640 1 --2- 172.21.15.92:0/1192268498 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f26200ce890 0x7f2620166690 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.674+0000 7f262759b640 1 -- 172.21.15.92:0/1192268498 >> 172.21.15.92:0/1192268498 conn(0x7f26200b65d0 msgr2=0x7f26200b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:57.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.674+0000 7f262759b640 1 -- 172.21.15.92:0/1192268498 shutdown_connections 2024-09-17T04:32:57.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.674+0000 7f262759b640 1 -- 172.21.15.92:0/1192268498 wait complete. 2024-09-17T04:32:57.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.677+0000 7f262759b640 1 Processor -- start 2024-09-17T04:32:57.677 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.677+0000 7f262759b640 1 -- start start 2024-09-17T04:32:57.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.679+0000 7f262759b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f26200ce890 0x7f2620143e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:57.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.679+0000 7f262759b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f262015b620 0x7f26201f7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:57.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.680+0000 7f262759b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26200b2270 0x7f262006cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:57.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.680+0000 7f262759b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2620166d50 con 0x7f262015b620 2024-09-17T04:32:57.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.680+0000 7f262759b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2620166bd0 con 0x7f26200b2270 2024-09-17T04:32:57.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.680+0000 7f262759b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f2620166ed0 con 0x7f26200ce890 2024-09-17T04:32:57.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.680+0000 7f2625337640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f26200ce890 0x7f2620143e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:57.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.680+0000 7f2625337640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f26200ce890 0x7f2620143e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.92:38186/0 (socket says 172.21.15.92:38186) 2024-09-17T04:32:57.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.680+0000 7f2625337640 1 -- 172.21.15.92:0/3761122496 learned_addr learned my addr 172.21.15.92:0/3761122496 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-17T04:32:57.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.680+0000 7f2624b36640 1 --2- 172.21.15.92:0/3761122496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f262015b620 0x7f26201f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:57.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.680+0000 7f2625b38640 1 --2- 172.21.15.92:0/3761122496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26200b2270 0x7f262006cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:57.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.680+0000 7f2625337640 1 -- 172.21.15.92:0/3761122496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26200b2270 msgr2=0x7f262006cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:57.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.680+0000 7f2625337640 1 --2- 172.21.15.92:0/3761122496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26200b2270 0x7f262006cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.680+0000 7f2625337640 1 -- 172.21.15.92:0/3761122496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f262015b620 msgr2=0x7f26201f7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:57.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.680+0000 7f2625337640 1 --2- 172.21.15.92:0/3761122496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f262015b620 0x7f26201f7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.680+0000 7f2625337640 1 -- 172.21.15.92:0/3761122496 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262015e6c0 con 0x7f26200ce890 2024-09-17T04:32:57.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.680+0000 7f2624b36640 1 --2- 172.21.15.92:0/3761122496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f262015b620 0x7f26201f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:32:57.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.680+0000 7f2625337640 1 --2- 172.21.15.92:0/3761122496 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f26200ce890 0x7f2620143e40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f261000c3c0 tx=0x7f261000c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:57.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.680+0000 7f2615ffb640 1 -- 172.21.15.92:0/3761122496 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610017070 con 0x7f26200ce890 2024-09-17T04:32:57.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.680+0000 7f2615ffb640 1 -- 172.21.15.92:0/3761122496 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26100053c0 con 0x7f26200ce890 2024-09-17T04:32:57.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.680+0000 7f2615ffb640 1 -- 172.21.15.92:0/3761122496 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610003850 con 0x7f26200ce890 2024-09-17T04:32:57.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.680+0000 7f262759b640 1 -- 172.21.15.92:0/3761122496 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f262015e9a0 con 0x7f26200ce890 2024-09-17T04:32:57.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.681+0000 7f2615ffb640 1 -- 172.21.15.92:0/3761122496 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f261001a070 con 0x7f26200ce890 2024-09-17T04:32:57.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.681+0000 7f262759b640 1 -- 172.21.15.92:0/3761122496 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f26201f6e20 con 0x7f26200ce890 2024-09-17T04:32:57.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.681+0000 7f2615ffb640 1 -- 172.21.15.92:0/3761122496 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f2610011cd0 con 0x7f26200ce890 2024-09-17T04:32:57.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.682+0000 7f262759b640 1 -- 172.21.15.92:0/3761122496 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f262006f670 con 0x7f26200ce890 2024-09-17T04:32:57.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.684+0000 7f2615ffb640 1 -- 172.21.15.92:0/3761122496 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f2610022020 con 0x7f26200ce890 2024-09-17T04:32:57.752 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.752+0000 7f262759b640 1 -- 172.21.15.92:0/3761122496 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd new", "uuid": "62f12540-d374-4a4c-9ff7-63b7304fe77f", "id": 7} v 0) -- 0x7f262014e050 con 0x7f26200ce890 2024-09-17T04:32:57.756 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.756+0000 7f2615ffb640 1 -- 172.21.15.92:0/3761122496 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "62f12540-d374-4a4c-9ff7-63b7304fe77f", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f2610011590 con 0x7f26200ce890 2024-09-17T04:32:57.756 INFO:teuthology.orchestra.run.smithi092.stdout:7 2024-09-17T04:32:57.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.757+0000 7f262759b640 1 -- 172.21.15.92:0/3761122496 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f26200ce890 msgr2=0x7f2620143e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:57.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.757+0000 7f262759b640 1 --2- 172.21.15.92:0/3761122496 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f26200ce890 0x7f2620143e40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f261000c3c0 tx=0x7f261000c880 comp rx=0 tx=0).stop 2024-09-17T04:32:57.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.757+0000 7f262759b640 1 -- 172.21.15.92:0/3761122496 shutdown_connections 2024-09-17T04:32:57.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.757+0000 7f262759b640 1 --2- 172.21.15.92:0/3761122496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26200b2270 0x7f262006cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.757 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.757+0000 7f262759b640 1 --2- 172.21.15.92:0/3761122496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f262015b620 0x7f26201f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.757+0000 7f262759b640 1 --2- 172.21.15.92:0/3761122496 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f26200ce890 0x7f2620143e40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.757+0000 7f262759b640 1 -- 172.21.15.92:0/3761122496 >> 172.21.15.92:0/3761122496 conn(0x7f26200b65d0 msgr2=0x7f2620150da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:57.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.757+0000 7f262759b640 1 -- 172.21.15.92:0/3761122496 shutdown_connections 2024-09-17T04:32:57.758 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:32:57.757+0000 7f262759b640 1 -- 172.21.15.92:0/3761122496 wait complete. 2024-09-17T04:32:57.767 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-17T04:32:57.768 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-17T04:32:57.770 INFO:tasks.ceph.osd.0:Started 2024-09-17T04:32:57.770 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-17T04:32:57.770 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-17T04:32:57.772 INFO:tasks.ceph.osd.1:Started 2024-09-17T04:32:57.772 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-17T04:32:57.772 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-17T04:32:57.774 INFO:tasks.ceph.osd.2:Started 2024-09-17T04:32:57.775 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-17T04:32:57.775 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-17T04:32:57.777 INFO:tasks.ceph.osd.3:Started 2024-09-17T04:32:57.777 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-17T04:32:57.777 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-17T04:32:57.779 INFO:tasks.ceph.osd.4:Started 2024-09-17T04:32:57.779 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-17T04:32:57.780 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-17T04:32:57.781 INFO:tasks.ceph.osd.5:Started 2024-09-17T04:32:57.781 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-17T04:32:57.782 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-17T04:32:57.783 INFO:tasks.ceph.osd.6:Started 2024-09-17T04:32:57.784 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-17T04:32:57.784 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-17T04:32:57.786 INFO:tasks.ceph.osd.7:Started 2024-09-17T04:32:57.786 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:32:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.931+0000 7fcd5549d640 1 -- 172.21.15.37:0/823933581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5015a310 msgr2=0x7fcd50160130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.931+0000 7fcd5549d640 1 --2- 172.21.15.37:0/823933581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5015a310 0x7fcd50160130 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcd50074e70 tx=0x7fcd4002a560 comp rx=0 tx=0).stop 2024-09-17T04:32:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.932+0000 7fcd5549d640 1 -- 172.21.15.37:0/823933581 shutdown_connections 2024-09-17T04:32:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.932+0000 7fcd5549d640 1 --2- 172.21.15.37:0/823933581 >> v2:172.21.15.92:3300/0 conn(0x7fcd500d03e0 0x7fcd5015d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.932+0000 7fcd5549d640 1 --2- 172.21.15.37:0/823933581 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcd500b7d90 0x7fcd500cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.932+0000 7fcd5549d640 1 --2- 172.21.15.37:0/823933581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5015a310 0x7fcd50160130 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.932+0000 7fcd5549d640 1 -- 172.21.15.37:0/823933581 >> 172.21.15.37:0/823933581 conn(0x7fcd50069330 msgr2=0x7fcd50071060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.932+0000 7fcd5549d640 1 -- 172.21.15.37:0/823933581 shutdown_connections 2024-09-17T04:32:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.932+0000 7fcd5549d640 1 -- 172.21.15.37:0/823933581 wait complete. 2024-09-17T04:32:57.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.934+0000 7fcd5549d640 1 Processor -- start 2024-09-17T04:32:57.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.934+0000 7fcd5549d640 1 -- start start 2024-09-17T04:32:57.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.935+0000 7fcd5549d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd500b7d90 0x7fcd5006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:57.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.935+0000 7fcd5549d640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcd500d03e0 0x7fcd500c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:57.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.935+0000 7fcd5549d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5015a310 0x7fcd50158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:57.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.935+0000 7fcd5549d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcd500cece0 con 0x7fcd5015a310 2024-09-17T04:32:57.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.935+0000 7fcd4e7fc640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcd500d03e0 0x7fcd500c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:57.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.935+0000 7fcd4effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd500b7d90 0x7fcd5006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:57.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.935+0000 7fcd4f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5015a310 0x7fcd50158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:57.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.935+0000 7fcd4e7fc640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcd500d03e0 0x7fcd500c9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.37:57158/0 (socket says 172.21.15.37:57158) 2024-09-17T04:32:57.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.935+0000 7fcd4effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd500b7d90 0x7fcd5006e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.37:51784/0 (socket says 172.21.15.37:51784) 2024-09-17T04:32:57.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.935+0000 7fcd4e7fc640 1 -- 172.21.15.37:0/2396470039 learned_addr learned my addr 172.21.15.37:0/2396470039 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:32:57.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.935+0000 7fcd5549d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5015da00 con 0x7fcd500b7d90 2024-09-17T04:32:57.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.935+0000 7fcd5549d640 1 -- 172.21.15.37:0/2396470039 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fcd500cee60 con 0x7fcd500d03e0 2024-09-17T04:32:57.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.935+0000 7fcd4effd640 1 -- 172.21.15.37:0/2396470039 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcd500d03e0 msgr2=0x7fcd500c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:57.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.935+0000 7fcd4effd640 1 --2- 172.21.15.37:0/2396470039 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcd500d03e0 0x7fcd500c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.935+0000 7fcd4effd640 1 -- 172.21.15.37:0/2396470039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5015a310 msgr2=0x7fcd50158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:57.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.935+0000 7fcd4effd640 1 --2- 172.21.15.37:0/2396470039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5015a310 0x7fcd50158930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:57.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.935+0000 7fcd4effd640 1 -- 172.21.15.37:0/2396470039 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd500c8680 con 0x7fcd500b7d90 2024-09-17T04:32:57.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.935+0000 7fcd4e7fc640 1 --2- 172.21.15.37:0/2396470039 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcd500d03e0 0x7fcd500c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:32:57.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.936+0000 7fcd4effd640 1 --2- 172.21.15.37:0/2396470039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd500b7d90 0x7fcd5006e090 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcd500cfe50 tx=0x7fcd400038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:57.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.936+0000 7fcd377fe640 1 -- 172.21.15.37:0/2396470039 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4003e030 con 0x7fcd500b7d90 2024-09-17T04:32:57.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.936+0000 7fcd377fe640 1 -- 172.21.15.37:0/2396470039 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd40003ca0 con 0x7fcd500b7d90 2024-09-17T04:32:57.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.936+0000 7fcd5549d640 1 -- 172.21.15.37:0/2396470039 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd500c8960 con 0x7fcd500b7d90 2024-09-17T04:32:57.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.936+0000 7fcd377fe640 1 -- 172.21.15.37:0/2396470039 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40035040 con 0x7fcd500b7d90 2024-09-17T04:32:57.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.936+0000 7fcd5549d640 1 -- 172.21.15.37:0/2396470039 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd5006e5d0 con 0x7fcd500b7d90 2024-09-17T04:32:57.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.936+0000 7fcd377fe640 1 -- 172.21.15.37:0/2396470039 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcd40041030 con 0x7fcd500b7d90 2024-09-17T04:32:57.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.936+0000 7fcd377fe640 1 -- 172.21.15.37:0/2396470039 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fcd40031ad0 con 0x7fcd500b7d90 2024-09-17T04:32:57.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.937+0000 7fcd5549d640 1 -- 172.21.15.37:0/2396470039 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd5006e800 con 0x7fcd500b7d90 2024-09-17T04:32:57.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:57.939+0000 7fcd377fe640 1 -- 172.21.15.37:0/2396470039 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fcd5006e800 con 0x7fcd500b7d90 2024-09-17T04:32:58.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:58.002+0000 7fcd5549d640 1 -- 172.21.15.37:0/2396470039 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcd500cf380 con 0x7fcd500b7d90 2024-09-17T04:32:58.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:58.002+0000 7fcd377fe640 1 -- 172.21.15.37:0/2396470039 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fcd40004f70 con 0x7fcd500b7d90 2024-09-17T04:32:58.003 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:32:58.003 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":10,"fsid":"a908a603-430c-4209-8294-0c3460b35f42","created":"2024-09-17T04:32:54.139221+0000","modified":"2024-09-17T04:32:57.754118+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T04:32:57.754118+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6ea4b8d1-0d22-4dd9-b913-9df585ef24ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ac745a33-e8ad-4083-9c18-71bf81982b76","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3c63f3d8-132a-4735-b0bc-d64313a46503","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4f8c6ca2-ea1d-4610-9127-52c4dd8bc8e3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"63d0c4ab-f115-47b4-9611-12a10153473a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4c56c04d-0c07-4f76-9d75-6335121310ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b30eae7c-096a-4d24-ac51-96b8b13ecaee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"62f12540-d374-4a4c-9ff7-63b7304fe77f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:32:58.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:58.003+0000 7fcd5549d640 1 -- 172.21.15.37:0/2396470039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd500b7d90 msgr2=0x7fcd5006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:58.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:58.003+0000 7fcd5549d640 1 --2- 172.21.15.37:0/2396470039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd500b7d90 0x7fcd5006e090 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcd500cfe50 tx=0x7fcd400038a0 comp rx=0 tx=0).stop 2024-09-17T04:32:58.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:58.003+0000 7fcd5549d640 1 -- 172.21.15.37:0/2396470039 shutdown_connections 2024-09-17T04:32:58.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:58.003+0000 7fcd5549d640 1 --2- 172.21.15.37:0/2396470039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5015a310 0x7fcd50158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:58.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:58.003+0000 7fcd5549d640 1 --2- 172.21.15.37:0/2396470039 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcd500d03e0 0x7fcd500c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:58.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:58.003+0000 7fcd5549d640 1 --2- 172.21.15.37:0/2396470039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd500b7d90 0x7fcd5006e090 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:58.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:58.003+0000 7fcd5549d640 1 -- 172.21.15.37:0/2396470039 >> 172.21.15.37:0/2396470039 conn(0x7fcd50069330 msgr2=0x7fcd5015f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:58.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:58.003+0000 7fcd5549d640 1 -- 172.21.15.37:0/2396470039 shutdown_connections 2024-09-17T04:32:58.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:32:58.003+0000 7fcd5549d640 1 -- 172.21.15.37:0/2396470039 wait complete. 2024-09-17T04:32:58.015 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-17T04:32:58.015 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-17T04:32:58.784 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-17T04:32:58.784+0000 7fb559064640 -1 Falling back to public interface 2024-09-17T04:32:58.821 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T04:32:58.930 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.928+0000 7fc53f7ba640 1 -- 172.21.15.37:0/1909744297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5380b6720 msgr2=0x7fc538160cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:58.930 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.928+0000 7fc53f7ba640 1 --2- 172.21.15.37:0/1909744297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5380b6720 0x7fc538160cb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc5381609a0 tx=0x7fc52c02a5c0 comp rx=0 tx=0).stop 2024-09-17T04:32:58.930 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.928+0000 7fc53f7ba640 1 -- 172.21.15.37:0/1909744297 shutdown_connections 2024-09-17T04:32:58.930 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.928+0000 7fc53f7ba640 1 --2- 172.21.15.37:0/1909744297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5380b6720 0x7fc538160cb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:58.930 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.928+0000 7fc53f7ba640 1 -- 172.21.15.37:0/1909744297 >> 172.21.15.37:0/1909744297 conn(0x7fc53809aa20 msgr2=0x7fc5380b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:58.930 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.929+0000 7fc53f7ba640 1 -- 172.21.15.37:0/1909744297 shutdown_connections 2024-09-17T04:32:58.930 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.929+0000 7fc53f7ba640 1 -- 172.21.15.37:0/1909744297 wait complete. 2024-09-17T04:32:58.931 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.930+0000 7fc53f7ba640 1 Processor -- start 2024-09-17T04:32:58.931 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.931+0000 7fc53f7ba640 1 -- start start 2024-09-17T04:32:58.934 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.932+0000 7fc53f7ba640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5380b36d0 0x7fc538158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:58.934 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.932+0000 7fc53f7ba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5380b6720 0x7fc5380b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:58.935 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.932+0000 7fc53f7ba640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc5380b6b30 0x7fc5380b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:32:58.935 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.932+0000 7fc53f7ba640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc53815d1f0 con 0x7fc5380b6720 2024-09-17T04:32:58.935 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.932+0000 7fc53f7ba640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc53815d070 con 0x7fc5380b36d0 2024-09-17T04:32:58.935 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.932+0000 7fc53f7ba640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fc53815d370 con 0x7fc5380b6b30 2024-09-17T04:32:58.935 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.932+0000 7fc53cd55640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5380b6720 0x7fc5380b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:58.935 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.932+0000 7fc53cd55640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5380b6720 0x7fc5380b56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46266/0 (socket says 172.21.15.37:46266) 2024-09-17T04:32:58.935 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.932+0000 7fc53cd55640 1 -- 172.21.15.37:0/2600174787 learned_addr learned my addr 172.21.15.37:0/2600174787 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:32:58.935 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.932+0000 7fc53dd57640 1 --2- 172.21.15.37:0/2600174787 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc5380b6b30 0x7fc5380b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:58.935 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.932+0000 7fc53d556640 1 --2- 172.21.15.37:0/2600174787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5380b36d0 0x7fc538158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:32:58.935 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.932+0000 7fc53cd55640 1 -- 172.21.15.37:0/2600174787 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc5380b6b30 msgr2=0x7fc5380b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:58.935 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.932+0000 7fc53cd55640 1 --2- 172.21.15.37:0/2600174787 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc5380b6b30 0x7fc5380b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:58.935 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.932+0000 7fc53cd55640 1 -- 172.21.15.37:0/2600174787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5380b36d0 msgr2=0x7fc538158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:58.936 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.932+0000 7fc53cd55640 1 --2- 172.21.15.37:0/2600174787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5380b36d0 0x7fc538158d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:58.936 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.932+0000 7fc53cd55640 1 -- 172.21.15.37:0/2600174787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc538159270 con 0x7fc5380b6720 2024-09-17T04:32:58.936 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.932+0000 7fc53d556640 1 --2- 172.21.15.37:0/2600174787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5380b36d0 0x7fc538158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:32:58.936 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.933+0000 7fc53cd55640 1 --2- 172.21.15.37:0/2600174787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5380b6720 0x7fc5380b56a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc52c02abb0 tx=0x7fc52c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:32:58.936 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.933+0000 7fc529ffb640 1 -- 172.21.15.37:0/2600174787 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c0044e0 con 0x7fc5380b6720 2024-09-17T04:32:58.936 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.933+0000 7fc529ffb640 1 -- 172.21.15.37:0/2600174787 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc52c005310 con 0x7fc5380b6720 2024-09-17T04:32:58.936 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.933+0000 7fc53f7ba640 1 -- 172.21.15.37:0/2600174787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5381f5810 con 0x7fc5380b6720 2024-09-17T04:32:58.936 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.933+0000 7fc529ffb640 1 -- 172.21.15.37:0/2600174787 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c003800 con 0x7fc5380b6720 2024-09-17T04:32:58.936 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.933+0000 7fc53f7ba640 1 -- 172.21.15.37:0/2600174787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5381f5dd0 con 0x7fc5380b6720 2024-09-17T04:32:58.937 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.934+0000 7fc53f7ba640 1 -- 172.21.15.37:0/2600174787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc538050f60 con 0x7fc5380b6720 2024-09-17T04:32:58.937 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.934+0000 7fc529ffb640 1 -- 172.21.15.37:0/2600174787 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 2) ==== 50742+0+0 (secure 0 0 0) 0x7fc52c0039a0 con 0x7fc5380b6720 2024-09-17T04:32:58.937 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.934+0000 7fc529ffb640 1 -- 172.21.15.37:0/2600174787 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fc52c073880 con 0x7fc5380b6720 2024-09-17T04:32:58.939 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:58.938+0000 7fc529ffb640 1 -- 172.21.15.37:0/2600174787 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fc538050f60 con 0x7fc5380b6720 2024-09-17T04:32:59.001 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:59.000+0000 7fc53f7ba640 1 -- 172.21.15.37:0/2600174787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc53815d7f0 con 0x7fc5380b6720 2024-09-17T04:32:59.002 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:59.001+0000 7fc529ffb640 1 -- 172.21.15.37:0/2600174787 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fc52c03f030 con 0x7fc5380b6720 2024-09-17T04:32:59.002 INFO:teuthology.misc.health.smithi037.stdout: 2024-09-17T04:32:59.002 INFO:teuthology.misc.health.smithi037.stdout:{"epoch":10,"fsid":"a908a603-430c-4209-8294-0c3460b35f42","created":"2024-09-17T04:32:54.139221+0000","modified":"2024-09-17T04:32:57.754118+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T04:32:57.754118+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6ea4b8d1-0d22-4dd9-b913-9df585ef24ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ac745a33-e8ad-4083-9c18-71bf81982b76","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3c63f3d8-132a-4735-b0bc-d64313a46503","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4f8c6ca2-ea1d-4610-9127-52c4dd8bc8e3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"63d0c4ab-f115-47b4-9611-12a10153473a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4c56c04d-0c07-4f76-9d75-6335121310ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b30eae7c-096a-4d24-ac51-96b8b13ecaee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"62f12540-d374-4a4c-9ff7-63b7304fe77f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:32:59.003 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:59.002+0000 7fc53f7ba640 1 -- 172.21.15.37:0/2600174787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5380b6720 msgr2=0x7fc5380b56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:32:59.003 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:59.002+0000 7fc53f7ba640 1 --2- 172.21.15.37:0/2600174787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5380b6720 0x7fc5380b56a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc52c02abb0 tx=0x7fc52c0042f0 comp rx=0 tx=0).stop 2024-09-17T04:32:59.003 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:59.002+0000 7fc53f7ba640 1 -- 172.21.15.37:0/2600174787 shutdown_connections 2024-09-17T04:32:59.003 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:59.002+0000 7fc53f7ba640 1 --2- 172.21.15.37:0/2600174787 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc5380b6b30 0x7fc5380b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:59.003 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:59.002+0000 7fc53f7ba640 1 --2- 172.21.15.37:0/2600174787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5380b6720 0x7fc5380b56a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:59.003 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:59.002+0000 7fc53f7ba640 1 --2- 172.21.15.37:0/2600174787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5380b36d0 0x7fc538158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:32:59.003 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:59.002+0000 7fc53f7ba640 1 -- 172.21.15.37:0/2600174787 >> 172.21.15.37:0/2600174787 conn(0x7fc53809aa20 msgr2=0x7fc5380b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:32:59.003 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:59.002+0000 7fc53f7ba640 1 -- 172.21.15.37:0/2600174787 shutdown_connections 2024-09-17T04:32:59.004 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:32:59.002+0000 7fc53f7ba640 1 -- 172.21.15.37:0/2600174787 wait complete. 2024-09-17T04:32:59.012 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-17T04:32:59.024 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-17T04:32:59.024+0000 7fce5a815640 -1 Falling back to public interface 2024-09-17T04:32:59.042 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T04:32:59.041+0000 7f7771062640 -1 Falling back to public interface 2024-09-17T04:32:59.239 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-17T04:32:59.240+0000 7f49ba60c640 -1 Falling back to public interface 2024-09-17T04:32:59.252 INFO:tasks.ceph.osd.0.smithi037.stderr:2024-09-17T04:32:59.252+0000 7fb170e0c640 -1 Falling back to public interface 2024-09-17T04:32:59.280 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-17T04:32:59.280+0000 7fb0ebce8640 -1 Falling back to public interface 2024-09-17T04:32:59.281 INFO:tasks.ceph.osd.2.smithi037.stderr:2024-09-17T04:32:59.280+0000 7f44c9419640 -1 Falling back to public interface 2024-09-17T04:32:59.294 INFO:tasks.ceph.osd.1.smithi037.stderr:2024-09-17T04:32:59.294+0000 7f163801e640 -1 Falling back to public interface 2024-09-17T04:32:59.449 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-09-17T04:32:59.450+0000 7f9f878ae640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-17T04:32:59.450 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-09-17T04:32:59.450+0000 7f9f878ae640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-17T04:33:02.811 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-17T04:33:02.811+0000 7fce5a815640 -1 osd.6 0 log_to_monitors true 2024-09-17T04:33:02.841 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-17T04:33:02.841+0000 7fb559064640 -1 osd.7 0 log_to_monitors true 2024-09-17T04:33:03.017 INFO:tasks.ceph.osd.0.smithi037.stderr:2024-09-17T04:33:03.016+0000 7fb170e0c640 -1 osd.0 0 log_to_monitors true 2024-09-17T04:33:03.022 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-17T04:33:03.022+0000 7fb0ebce8640 -1 osd.5 0 log_to_monitors true 2024-09-17T04:33:03.066 INFO:tasks.ceph.osd.2.smithi037.stderr:2024-09-17T04:33:03.065+0000 7f44c9419640 -1 osd.2 0 log_to_monitors true 2024-09-17T04:33:03.107 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T04:33:03.107+0000 7f7771062640 -1 osd.3 0 log_to_monitors true 2024-09-17T04:33:03.129 INFO:tasks.ceph.osd.1.smithi037.stderr:2024-09-17T04:33:03.128+0000 7f163801e640 -1 osd.1 0 log_to_monitors true 2024-09-17T04:33:03.228 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-17T04:33:03.229+0000 7f49ba60c640 -1 osd.4 0 log_to_monitors true 2024-09-17T04:33:05.820 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T04:33:05.915 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-17T04:33:05.915+0000 7fce56125640 -1 osd.6 0 waiting for initial osdmap 2024-09-17T04:33:05.936 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-09-17T04:33:05.937+0000 7fce4df1b640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T04:33:05.960 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-17T04:33:05.960+0000 7fb554ff2640 -1 osd.7 0 waiting for initial osdmap 2024-09-17T04:33:05.974 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-09-17T04:33:05.975+0000 7fb54cde8640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T04:33:05.989 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.986+0000 7f9157e07640 1 -- 172.21.15.37:0/3917088066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f915006fb90 msgr2=0x7f91500aefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:05.989 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.986+0000 7f9157e07640 1 --2- 172.21.15.37:0/3917088066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f915006fb90 0x7f91500aefa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f915006ef60 tx=0x7f914c02b7d0 comp rx=0 tx=0).stop 2024-09-17T04:33:05.990 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.987+0000 7f9157e07640 1 -- 172.21.15.37:0/3917088066 shutdown_connections 2024-09-17T04:33:05.990 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.987+0000 7f9157e07640 1 --2- 172.21.15.37:0/3917088066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f915006fb90 0x7f91500aefa0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:05.990 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.987+0000 7f9157e07640 1 --2- 172.21.15.37:0/3917088066 >> v2:172.21.15.37:3300/0 conn(0x7f9150070ec0 0x7f91500ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:05.990 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.987+0000 7f9157e07640 1 -- 172.21.15.37:0/3917088066 >> 172.21.15.37:0/3917088066 conn(0x7f91500b1ca0 msgr2=0x7f91500b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:05.990 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.987+0000 7f9157e07640 1 -- 172.21.15.37:0/3917088066 shutdown_connections 2024-09-17T04:33:05.990 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.987+0000 7f9157e07640 1 -- 172.21.15.37:0/3917088066 wait complete. 2024-09-17T04:33:05.991 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.990+0000 7f9157e07640 1 Processor -- start 2024-09-17T04:33:05.991 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.990+0000 7f9157e07640 1 -- start start 2024-09-17T04:33:05.994 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.992+0000 7f9157e07640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f915006fb90 0x7f915009c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:05.994 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.992+0000 7f9157e07640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9150070ec0 0x7f91500ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:05.995 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.992+0000 7f9157e07640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91500cd020 0x7f91500ae620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:05.995 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.993+0000 7f91563a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91500cd020 0x7f91500ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:05.995 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.993+0000 7f91563a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91500cd020 0x7f91500ae620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32862/0 (socket says 172.21.15.37:32862) 2024-09-17T04:33:05.995 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.993+0000 7f9155ba3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f915006fb90 0x7f915009c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:05.995 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.993+0000 7f91563a4640 1 -- 172.21.15.37:0/2477827387 learned_addr learned my addr 172.21.15.37:0/2477827387 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:05.995 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.993+0000 7f91553a2640 1 --2- 172.21.15.37:0/2477827387 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9150070ec0 0x7f91500ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:05.995 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.993+0000 7f9157e07640 1 -- 172.21.15.37:0/2477827387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f91500af870 con 0x7f91500cd020 2024-09-17T04:33:05.995 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.993+0000 7f9157e07640 1 -- 172.21.15.37:0/2477827387 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f91500af6f0 con 0x7f915006fb90 2024-09-17T04:33:05.995 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.993+0000 7f9157e07640 1 -- 172.21.15.37:0/2477827387 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f91500af9f0 con 0x7f9150070ec0 2024-09-17T04:33:05.996 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.994+0000 7f91563a4640 1 -- 172.21.15.37:0/2477827387 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9150070ec0 msgr2=0x7f91500ae0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:05.996 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.994+0000 7f91563a4640 1 --2- 172.21.15.37:0/2477827387 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9150070ec0 0x7f91500ae0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:05.997 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.994+0000 7f91563a4640 1 -- 172.21.15.37:0/2477827387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f915006fb90 msgr2=0x7f915009c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:05.997 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.994+0000 7f91563a4640 1 --2- 172.21.15.37:0/2477827387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f915006fb90 0x7f915009c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:05.997 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.994+0000 7f91563a4640 1 -- 172.21.15.37:0/2477827387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f915014e650 con 0x7f91500cd020 2024-09-17T04:33:05.997 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.994+0000 7f91563a4640 1 --2- 172.21.15.37:0/2477827387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91500cd020 0x7f91500ae620 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f914c00aa30 tx=0x7f914c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:05.997 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.994+0000 7f9155ba3640 1 --2- 172.21.15.37:0/2477827387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f915006fb90 0x7f915009c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:05.997 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.994+0000 7f91553a2640 1 --2- 172.21.15.37:0/2477827387 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9150070ec0 0x7f91500ae0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:05.997 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.995+0000 7f91467fc640 1 -- 172.21.15.37:0/2477827387 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914c042020 con 0x7f91500cd020 2024-09-17T04:33:05.998 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.995+0000 7f91467fc640 1 -- 172.21.15.37:0/2477827387 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f914c03a040 con 0x7f91500cd020 2024-09-17T04:33:05.998 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.995+0000 7f91467fc640 1 -- 172.21.15.37:0/2477827387 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914c039070 con 0x7f91500cd020 2024-09-17T04:33:05.998 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.995+0000 7f9157e07640 1 -- 172.21.15.37:0/2477827387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f915009cd70 con 0x7f91500cd020 2024-09-17T04:33:05.999 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.995+0000 7f9157e07640 1 -- 172.21.15.37:0/2477827387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f915014eda0 con 0x7f91500cd020 2024-09-17T04:33:05.999 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:05.997+0000 7f9157e07640 1 -- 172.21.15.37:0/2477827387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f915014e820 con 0x7f91500cd020 2024-09-17T04:33:06.001 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:06.000+0000 7f91467fc640 1 -- 172.21.15.37:0/2477827387 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) ==== 101385+0+0 (secure 0 0 0) 0x7f914c045020 con 0x7f91500cd020 2024-09-17T04:33:06.002 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:06.001+0000 7f91467fc640 1 --2- 172.21.15.37:0/2477827387 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9134078c00 0x7f913407b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:06.002 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:06.001+0000 7f91467fc640 1 -- 172.21.15.37:0/2477827387 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2231+0+0 (secure 0 0 0) 0x7f914c0bafb0 con 0x7f91500cd020 2024-09-17T04:33:06.002 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:06.001+0000 7f9155ba3640 1 --2- 172.21.15.37:0/2477827387 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9134078c00 0x7f913407b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:06.002 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:06.002+0000 7f9155ba3640 1 --2- 172.21.15.37:0/2477827387 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9134078c00 0x7f913407b110 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9148005fd0 tx=0x7f9148009500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:06.008 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:06.007+0000 7f91467fc640 1 -- 172.21.15.37:0/2477827387 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f914c0ebd90 con 0x7f91500cd020 2024-09-17T04:33:06.053 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-17T04:33:06.053+0000 7fb0e7c76640 -1 osd.5 0 waiting for initial osdmap 2024-09-17T04:33:06.070 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-09-17T04:33:06.071+0000 7fb0dfa6c640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T04:33:06.337 INFO:tasks.ceph.osd.2.smithi037.stderr:2024-09-17T04:33:06.336+0000 7f44c4e00640 -1 osd.2 0 waiting for initial osdmap 2024-09-17T04:33:06.356 INFO:tasks.ceph.osd.2.smithi037.stderr:2024-09-17T04:33:06.355+0000 7f44bcbf6640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T04:33:06.379 INFO:tasks.ceph.osd.1.smithi037.stderr:2024-09-17T04:33:06.378+0000 7f1633a5d640 -1 osd.1 0 waiting for initial osdmap 2024-09-17T04:33:06.380 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:06.379+0000 7f9157e07640 1 -- 172.21.15.37:0/2477827387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91500aefa0 con 0x7f91500cd020 2024-09-17T04:33:06.381 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:06.380+0000 7f91467fc640 1 -- 172.21.15.37:0/2477827387 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+7058 (secure 0 0 0) 0x7f915014e820 con 0x7f91500cd020 2024-09-17T04:33:06.381 INFO:teuthology.misc.health.smithi037.stdout: 2024-09-17T04:33:06.381 INFO:teuthology.misc.health.smithi037.stdout:{"epoch":13,"fsid":"a908a603-430c-4209-8294-0c3460b35f42","created":"2024-09-17T04:32:54.139221+0000","modified":"2024-09-17T04:33:05.188822+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T04:32:57.754118+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6ea4b8d1-0d22-4dd9-b913-9df585ef24ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ac745a33-e8ad-4083-9c18-71bf81982b76","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3c63f3d8-132a-4735-b0bc-d64313a46503","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4f8c6ca2-ea1d-4610-9127-52c4dd8bc8e3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"63d0c4ab-f115-47b4-9611-12a10153473a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4c56c04d-0c07-4f76-9d75-6335121310ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b30eae7c-096a-4d24-ac51-96b8b13ecaee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"62f12540-d374-4a4c-9ff7-63b7304fe77f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:33:06.384 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:06.383+0000 7f9157e07640 1 -- 172.21.15.37:0/2477827387 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9134078c00 msgr2=0x7f913407b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:06.384 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:06.383+0000 7f9157e07640 1 --2- 172.21.15.37:0/2477827387 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9134078c00 0x7f913407b110 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9148005fd0 tx=0x7f9148009500 comp rx=0 tx=0).stop 2024-09-17T04:33:06.384 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:06.383+0000 7f9157e07640 1 -- 172.21.15.37:0/2477827387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91500cd020 msgr2=0x7f91500ae620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:06.384 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:06.383+0000 7f9157e07640 1 --2- 172.21.15.37:0/2477827387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91500cd020 0x7f91500ae620 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f914c00aa30 tx=0x7f914c003820 comp rx=0 tx=0).stop 2024-09-17T04:33:06.385 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:06.384+0000 7f9157e07640 1 -- 172.21.15.37:0/2477827387 shutdown_connections 2024-09-17T04:33:06.388 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:06.384+0000 7f9157e07640 1 --2- 172.21.15.37:0/2477827387 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9134078c00 0x7f913407b110 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:06.388 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:06.384+0000 7f9157e07640 1 --2- 172.21.15.37:0/2477827387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91500cd020 0x7f91500ae620 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:06.388 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:06.384+0000 7f9157e07640 1 --2- 172.21.15.37:0/2477827387 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9150070ec0 0x7f91500ae0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:06.389 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:06.384+0000 7f9157e07640 1 --2- 172.21.15.37:0/2477827387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f915006fb90 0x7f915009c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:06.389 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:06.384+0000 7f9157e07640 1 -- 172.21.15.37:0/2477827387 >> 172.21.15.37:0/2477827387 conn(0x7f91500b1ca0 msgr2=0x7f91501726c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:06.389 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:06.385+0000 7f9157e07640 1 -- 172.21.15.37:0/2477827387 shutdown_connections 2024-09-17T04:33:06.389 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:06.385+0000 7f9157e07640 1 -- 172.21.15.37:0/2477827387 wait complete. 2024-09-17T04:33:06.395 INFO:tasks.ceph.osd.1.smithi037.stderr:2024-09-17T04:33:06.395+0000 7f162b853640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T04:33:06.402 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-17T04:33:06.409 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T04:33:06.409+0000 7f776d805640 -1 osd.3 0 waiting for initial osdmap 2024-09-17T04:33:06.417 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-17T04:33:06.417+0000 7f49b67dc640 -1 osd.4 0 waiting for initial osdmap 2024-09-17T04:33:06.421 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T04:33:06.421+0000 7f7764de6640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T04:33:06.426 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-09-17T04:33:06.427+0000 7f49addbd640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T04:33:06.455 INFO:tasks.ceph.osd.0.smithi037.stderr:2024-09-17T04:33:06.454+0000 7fb16c685640 -1 osd.0 0 waiting for initial osdmap 2024-09-17T04:33:06.470 INFO:tasks.ceph.osd.0.smithi037.stderr:2024-09-17T04:33:06.471+0000 7fb16447b640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T04:33:13.209 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T04:33:13.332 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.331+0000 7f3efd840640 1 -- 172.21.15.37:0/1139055379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef8070650 msgr2=0x7f3ef80ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:13.332 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.331+0000 7f3efd840640 1 --2- 172.21.15.37:0/1139055379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef8070650 0x7f3ef80ceac0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3ef80afba0 tx=0x7f3eec02a5f0 comp rx=0 tx=0).stop 2024-09-17T04:33:13.332 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.331+0000 7f3efd840640 1 -- 172.21.15.37:0/1139055379 shutdown_connections 2024-09-17T04:33:13.332 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.331+0000 7f3efd840640 1 --2- 172.21.15.37:0/1139055379 >> v2:172.21.15.92:3300/0 conn(0x7f3ef80b52d0 0x7f3ef80ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:13.332 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.331+0000 7f3efd840640 1 --2- 172.21.15.37:0/1139055379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef8070650 0x7f3ef80ceac0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:13.332 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.331+0000 7f3efd840640 1 -- 172.21.15.37:0/1139055379 >> 172.21.15.37:0/1139055379 conn(0x7f3ef80b1ca0 msgr2=0x7f3ef80b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:13.332 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.331+0000 7f3efd840640 1 -- 172.21.15.37:0/1139055379 shutdown_connections 2024-09-17T04:33:13.333 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.331+0000 7f3efd840640 1 -- 172.21.15.37:0/1139055379 wait complete. 2024-09-17T04:33:13.333 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.333+0000 7f3efd840640 1 Processor -- start 2024-09-17T04:33:13.333 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.333+0000 7f3efd840640 1 -- start start 2024-09-17T04:33:13.336 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.335+0000 7f3efd840640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef80af1d0 0x7f3ef80c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:13.336 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.335+0000 7f3efd840640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3ef80b52d0 0x7f3ef815e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:13.336 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.335+0000 7f3efd840640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef815d3f0 0x7f3ef815d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:13.336 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.335+0000 7f3efd840640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef80cd1c0 con 0x7f3ef815d3f0 2024-09-17T04:33:13.336 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.335+0000 7f3efd840640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef80cd340 con 0x7f3ef80af1d0 2024-09-17T04:33:13.337 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.335+0000 7f3efd840640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f3ef80cd4c0 con 0x7f3ef80b52d0 2024-09-17T04:33:13.337 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.335+0000 7f3ef77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef815d3f0 0x7f3ef815d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:13.337 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.335+0000 7f3ef6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef80af1d0 0x7f3ef80c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:13.337 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.335+0000 7f3ef67fc640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3ef80b52d0 0x7f3ef815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:13.337 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.335+0000 7f3ef6ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef80af1d0 0x7f3ef80c76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.37:59454/0 (socket says 172.21.15.37:59454) 2024-09-17T04:33:13.337 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.335+0000 7f3ef77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef815d3f0 0x7f3ef815d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32882/0 (socket says 172.21.15.37:32882) 2024-09-17T04:33:13.337 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.335+0000 7f3ef6ffd640 1 -- 172.21.15.37:0/147760138 learned_addr learned my addr 172.21.15.37:0/147760138 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:13.337 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.335+0000 7f3ef77fe640 1 -- 172.21.15.37:0/147760138 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3ef80b52d0 msgr2=0x7f3ef815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:13.337 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.335+0000 7f3ef77fe640 1 --2- 172.21.15.37:0/147760138 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3ef80b52d0 0x7f3ef815e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:13.337 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.335+0000 7f3ef77fe640 1 -- 172.21.15.37:0/147760138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef80af1d0 msgr2=0x7f3ef80c76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:13.337 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.335+0000 7f3ef77fe640 1 --2- 172.21.15.37:0/147760138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef80af1d0 0x7f3ef80c76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:13.337 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.335+0000 7f3ef77fe640 1 -- 172.21.15.37:0/147760138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef815dd40 con 0x7f3ef815d3f0 2024-09-17T04:33:13.337 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.335+0000 7f3ef77fe640 1 --2- 172.21.15.37:0/147760138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef815d3f0 0x7f3ef815d800 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000e960 tx=0x7f3ef000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:13.338 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.335+0000 7f3edffff640 1 -- 172.21.15.37:0/147760138 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0004d20 con 0x7f3ef815d3f0 2024-09-17T04:33:13.338 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.336+0000 7f3efd840640 1 -- 172.21.15.37:0/147760138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef80c7c00 con 0x7f3ef815d3f0 2024-09-17T04:33:13.338 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.336+0000 7f3edffff640 1 -- 172.21.15.37:0/147760138 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ef000ca60 con 0x7f3ef815d3f0 2024-09-17T04:33:13.338 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.336+0000 7f3edffff640 1 -- 172.21.15.37:0/147760138 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0005020 con 0x7f3ef815d3f0 2024-09-17T04:33:13.338 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.336+0000 7f3efd840640 1 -- 172.21.15.37:0/147760138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef80c9a80 con 0x7f3ef815d3f0 2024-09-17T04:33:13.339 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.337+0000 7f3efd840640 1 -- 172.21.15.37:0/147760138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ef80c94f0 con 0x7f3ef815d3f0 2024-09-17T04:33:13.339 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.338+0000 7f3edffff640 1 -- 172.21.15.37:0/147760138 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3ef00051c0 con 0x7f3ef815d3f0 2024-09-17T04:33:13.339 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.338+0000 7f3edffff640 1 --2- 172.21.15.37:0/147760138 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3ee4078b90 0x7f3ee407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:13.339 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.338+0000 7f3edffff640 1 -- 172.21.15.37:0/147760138 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f3ef009b190 con 0x7f3ef815d3f0 2024-09-17T04:33:13.339 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.338+0000 7f3ef6ffd640 1 --2- 172.21.15.37:0/147760138 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3ee4078b90 0x7f3ee407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:13.339 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.339+0000 7f3ef6ffd640 1 --2- 172.21.15.37:0/147760138 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3ee4078b90 0x7f3ee407b0a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3ee80017e0 tx=0x7f3ee8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:13.342 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.343+0000 7f3edffff640 1 -- 172.21.15.37:0/147760138 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3ef815dd40 con 0x7f3ef815d3f0 2024-09-17T04:33:13.527 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.528+0000 7f3efd840640 1 -- 172.21.15.37:0/147760138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ef806a110 con 0x7f3ef815d3f0 2024-09-17T04:33:13.528 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.528+0000 7f3edffff640 1 -- 172.21.15.37:0/147760138 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12836 (secure 0 0 0) 0x7f3ef0016030 con 0x7f3ef815d3f0 2024-09-17T04:33:13.528 INFO:teuthology.misc.health.smithi037.stdout: 2024-09-17T04:33:13.529 INFO:teuthology.misc.health.smithi037.stdout:{"epoch":18,"fsid":"a908a603-430c-4209-8294-0c3460b35f42","created":"2024-09-17T04:32:54.139221+0000","modified":"2024-09-17T04:33:10.945841+0000","last_up_change":"2024-09-17T04:33:06.922210+0000","last_in_change":"2024-09-17T04:32:57.754118+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:33:08.461907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ea4b8d1-0d22-4dd9-b913-9df585ef24ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6809","nonce":2283587593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6811","nonce":2283587593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6815","nonce":2283587593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6813","nonce":2283587593}]},"public_addr":"172.21.15.37:6809/2283587593","cluster_addr":"172.21.15.37:6811/2283587593","heartbeat_back_addr":"172.21.15.37:6815/2283587593","heartbeat_front_addr":"172.21.15.37:6813/2283587593","state":["exists","up"]},{"osd":1,"uuid":"ac745a33-e8ad-4083-9c18-71bf81982b76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6825","nonce":2657955935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6827","nonce":2657955935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6831","nonce":2657955935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6829","nonce":2657955935}]},"public_addr":"172.21.15.37:6825/2657955935","cluster_addr":"172.21.15.37:6827/2657955935","heartbeat_back_addr":"172.21.15.37:6831/2657955935","heartbeat_front_addr":"172.21.15.37:6829/2657955935","state":["exists","up"]},{"osd":2,"uuid":"3c63f3d8-132a-4735-b0bc-d64313a46503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6817","nonce":579720176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6819","nonce":579720176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6823","nonce":579720176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6821","nonce":579720176}]},"public_addr":"172.21.15.37:6817/579720176","cluster_addr":"172.21.15.37:6819/579720176","heartbeat_back_addr":"172.21.15.37:6823/579720176","heartbeat_front_addr":"172.21.15.37:6821/579720176","state":["exists","up"]},{"osd":3,"uuid":"4f8c6ca2-ea1d-4610-9127-52c4dd8bc8e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6801","nonce":809975826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6803","nonce":809975826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6807","nonce":809975826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6805","nonce":809975826}]},"public_addr":"172.21.15.37:6801/809975826","cluster_addr":"172.21.15.37:6803/809975826","heartbeat_back_addr":"172.21.15.37:6807/809975826","heartbeat_front_addr":"172.21.15.37:6805/809975826","state":["exists","up"]},{"osd":4,"uuid":"63d0c4ab-f115-47b4-9611-12a10153473a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6819","nonce":2509785752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6821","nonce":2509785752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6825","nonce":2509785752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6823","nonce":2509785752}]},"public_addr":"172.21.15.92:6819/2509785752","cluster_addr":"172.21.15.92:6821/2509785752","heartbeat_back_addr":"172.21.15.92:6825/2509785752","heartbeat_front_addr":"172.21.15.92:6823/2509785752","state":["exists","up"]},{"osd":5,"uuid":"4c56c04d-0c07-4f76-9d75-6335121310ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6827","nonce":248263832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6829","nonce":248263832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6833","nonce":248263832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6831","nonce":248263832}]},"public_addr":"172.21.15.92:6827/248263832","cluster_addr":"172.21.15.92:6829/248263832","heartbeat_back_addr":"172.21.15.92:6833/248263832","heartbeat_front_addr":"172.21.15.92:6831/248263832","state":["exists","up"]},{"osd":6,"uuid":"b30eae7c-096a-4d24-ac51-96b8b13ecaee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6811","nonce":4078365985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6813","nonce":4078365985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6817","nonce":4078365985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6815","nonce":4078365985}]},"public_addr":"172.21.15.92:6811/4078365985","cluster_addr":"172.21.15.92:6813/4078365985","heartbeat_back_addr":"172.21.15.92:6817/4078365985","heartbeat_front_addr":"172.21.15.92:6815/4078365985","state":["exists","up"]},{"osd":7,"uuid":"62f12540-d374-4a4c-9ff7-63b7304fe77f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6803","nonce":3444787223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6805","nonce":3444787223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6809","nonce":3444787223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6807","nonce":3444787223}]},"public_addr":"172.21.15.92:6803/3444787223","cluster_addr":"172.21.15.92:6805/3444787223","heartbeat_back_addr":"172.21.15.92:6809/3444787223","heartbeat_front_addr":"172.21.15.92:6807/3444787223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.006420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.111135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.103325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.072656+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.200989+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.060455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.774773+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.865778+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:33:13.531 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.530+0000 7f3efd840640 1 -- 172.21.15.37:0/147760138 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3ee4078b90 msgr2=0x7f3ee407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:13.531 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.530+0000 7f3efd840640 1 --2- 172.21.15.37:0/147760138 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3ee4078b90 0x7f3ee407b0a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3ee80017e0 tx=0x7f3ee8009040 comp rx=0 tx=0).stop 2024-09-17T04:33:13.531 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.530+0000 7f3efd840640 1 -- 172.21.15.37:0/147760138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef815d3f0 msgr2=0x7f3ef815d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:13.531 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.530+0000 7f3efd840640 1 --2- 172.21.15.37:0/147760138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef815d3f0 0x7f3ef815d800 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000e960 tx=0x7f3ef000ed60 comp rx=0 tx=0).stop 2024-09-17T04:33:13.531 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.530+0000 7f3efd840640 1 -- 172.21.15.37:0/147760138 shutdown_connections 2024-09-17T04:33:13.531 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.530+0000 7f3efd840640 1 --2- 172.21.15.37:0/147760138 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3ee4078b90 0x7f3ee407b0a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:13.531 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.530+0000 7f3efd840640 1 --2- 172.21.15.37:0/147760138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef815d3f0 0x7f3ef815d800 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:13.531 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.530+0000 7f3efd840640 1 --2- 172.21.15.37:0/147760138 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3ef80b52d0 0x7f3ef815e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:13.531 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.530+0000 7f3efd840640 1 --2- 172.21.15.37:0/147760138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef80af1d0 0x7f3ef80c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:13.531 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.530+0000 7f3efd840640 1 -- 172.21.15.37:0/147760138 >> 172.21.15.37:0/147760138 conn(0x7f3ef80b1ca0 msgr2=0x7f3ef80b0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:13.531 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.531+0000 7f3efd840640 1 -- 172.21.15.37:0/147760138 shutdown_connections 2024-09-17T04:33:13.531 INFO:teuthology.misc.health.smithi037.stderr:2024-09-17T04:33:13.531+0000 7f3efd840640 1 -- 172.21.15.37:0/147760138 wait complete. 2024-09-17T04:33:13.542 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-17T04:33:13.542 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-17T04:33:13.542 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T04:33:13.542 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T04:33:13.544 INFO:tasks.ceph.mds.a:Started 2024-09-17T04:33:13.544 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T04:33:13.545 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T04:33:13.546 INFO:tasks.ceph.mds.c:Started 2024-09-17T04:33:13.546 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T04:33:13.547 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T04:33:13.549 INFO:tasks.ceph.mds.b:Started 2024-09-17T04:33:13.549 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T04:33:13.550 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T04:33:13.552 INFO:tasks.ceph.mds.d:Started 2024-09-17T04:33:13.552 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-17T04:33:13.552 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:33:13.635 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-09-17T04:33:13.635 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-09-17T04:33:13.653 INFO:tasks.ceph.mds.d.smithi092.stdout:starting mds.d at 2024-09-17T04:33:13.653 INFO:tasks.ceph.mds.b.smithi092.stdout:starting mds.b at 2024-09-17T04:33:13.653 INFO:tasks.ceph.mds.b.smithi092.stderr:2024-09-17T04:33:13.652+0000 7fc8e9dad640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-17T04:33:13.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.676+0000 7f9a46e38640 1 -- 172.21.15.37:0/4226233724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a400d1a10 msgr2=0x7f9a400ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:13.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.676+0000 7f9a46e38640 1 --2- 172.21.15.37:0/4226233724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a400d1a10 0x7f9a400ccb50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c005fd0 tx=0x7f9a2c02a4d0 comp rx=0 tx=0).stop 2024-09-17T04:33:13.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.676+0000 7f9a46e38640 1 -- 172.21.15.37:0/4226233724 shutdown_connections 2024-09-17T04:33:13.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.676+0000 7f9a46e38640 1 --2- 172.21.15.37:0/4226233724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a400d1a10 0x7f9a400ccb50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:13.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.676+0000 7f9a46e38640 1 -- 172.21.15.37:0/4226233724 >> 172.21.15.37:0/4226233724 conn(0x7f9a40071050 msgr2=0x7f9a40071460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:13.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.676+0000 7f9a46e38640 1 -- 172.21.15.37:0/4226233724 shutdown_connections 2024-09-17T04:33:13.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.676+0000 7f9a46e38640 1 -- 172.21.15.37:0/4226233724 wait complete. 2024-09-17T04:33:13.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.678+0000 7f9a46e38640 1 Processor -- start 2024-09-17T04:33:13.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.678+0000 7f9a46e38640 1 -- start start 2024-09-17T04:33:13.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.680+0000 7f9a46e38640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9a400cd090 0x7f9a40156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:13.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.680+0000 7f9a46e38640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a400d1a10 0x7f9a40147580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:13.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.680+0000 7f9a46e38640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a4015c830 0x7f9a40147ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:13.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.680+0000 7f9a46e38640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9a400b6440 con 0x7f9a400d1a10 2024-09-17T04:33:13.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.680+0000 7f9a46e38640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9a400b62c0 con 0x7f9a4015c830 2024-09-17T04:33:13.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.680+0000 7f9a46e38640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f9a400b65c0 con 0x7f9a400cd090 2024-09-17T04:33:13.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.680+0000 7f9a3ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a400d1a10 0x7f9a40147580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:13.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.680+0000 7f9a3ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a400d1a10 0x7f9a40147580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32928/0 (socket says 172.21.15.37:32928) 2024-09-17T04:33:13.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.680+0000 7f9a3ffff640 1 -- 172.21.15.37:0/578420974 learned_addr learned my addr 172.21.15.37:0/578420974 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:13.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.680+0000 7f9a44bd4640 1 --2- 172.21.15.37:0/578420974 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9a400cd090 0x7f9a40156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:13.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.680+0000 7f9a453d5640 1 --2- 172.21.15.37:0/578420974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a4015c830 0x7f9a40147ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:13.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.680+0000 7f9a44bd4640 1 -- 172.21.15.37:0/578420974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a4015c830 msgr2=0x7f9a40147ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:13.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.680+0000 7f9a44bd4640 1 --2- 172.21.15.37:0/578420974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a4015c830 0x7f9a40147ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:13.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.680+0000 7f9a44bd4640 1 -- 172.21.15.37:0/578420974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a400d1a10 msgr2=0x7f9a40147580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:13.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.680+0000 7f9a44bd4640 1 --2- 172.21.15.37:0/578420974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a400d1a10 0x7f9a40147580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:13.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.680+0000 7f9a44bd4640 1 -- 172.21.15.37:0/578420974 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a40148320 con 0x7f9a400cd090 2024-09-17T04:33:13.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.680+0000 7f9a453d5640 1 --2- 172.21.15.37:0/578420974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a4015c830 0x7f9a40147ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:33:13.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.680+0000 7f9a3ffff640 1 --2- 172.21.15.37:0/578420974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a400d1a10 0x7f9a40147580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:33:13.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.680+0000 7f9a44bd4640 1 --2- 172.21.15.37:0/578420974 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9a400cd090 0x7f9a40156210 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400c760 tx=0x7f9a3400cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:13.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.681+0000 7f9a3d7fa640 1 -- 172.21.15.37:0/578420974 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a34017070 con 0x7f9a400cd090 2024-09-17T04:33:13.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.681+0000 7f9a3d7fa640 1 -- 172.21.15.37:0/578420974 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a3400ce10 con 0x7f9a400cd090 2024-09-17T04:33:13.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.681+0000 7f9a3d7fa640 1 -- 172.21.15.37:0/578420974 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3400a8a0 con 0x7f9a400cd090 2024-09-17T04:33:13.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.681+0000 7f9a46e38640 1 -- 172.21.15.37:0/578420974 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a401f3e80 con 0x7f9a400cd090 2024-09-17T04:33:13.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.681+0000 7f9a46e38640 1 -- 172.21.15.37:0/578420974 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a401f3b70 con 0x7f9a400cd090 2024-09-17T04:33:13.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.682+0000 7f9a46e38640 1 -- 172.21.15.37:0/578420974 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a40075390 con 0x7f9a400cd090 2024-09-17T04:33:13.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.682+0000 7f9a3d7fa640 1 -- 172.21.15.37:0/578420974 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9a3401a070 con 0x7f9a400cd090 2024-09-17T04:33:13.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.683+0000 7f9a3d7fa640 1 --2- 172.21.15.37:0/578420974 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9a24078b90 0x7f9a2407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:13.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.683+0000 7f9a3d7fa640 1 -- 172.21.15.37:0/578420974 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f9a34099e20 con 0x7f9a400cd090 2024-09-17T04:33:13.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.683+0000 7f9a3ffff640 1 --2- 172.21.15.37:0/578420974 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9a24078b90 0x7f9a2407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:13.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.684+0000 7f9a3ffff640 1 --2- 172.21.15.37:0/578420974 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9a24078b90 0x7f9a2407b0a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c009230 tx=0x7f9a2c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:13.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.688+0000 7f9a3d7fa640 1 -- 172.21.15.37:0/578420974 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9a40075390 con 0x7f9a400cd090 2024-09-17T04:33:13.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.910+0000 7f9a46e38640 1 -- 172.21.15.37:0/578420974 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a400ccb50 con 0x7f9a400cd090 2024-09-17T04:33:13.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.911+0000 7f9a3d7fa640 1 -- 172.21.15.37:0/578420974 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1434 (secure 0 0 0) 0x7f9a400cd4a0 con 0x7f9a400cd090 2024-09-17T04:33:13.911 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:13.911 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":2,"btime":"2024-09-17T04:33:13:641258+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4401,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/3514084301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3514084301},{"type":"v1","addr":"172.21.15.37:6834","nonce":3514084301}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-17T04:33:13.911 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 2 2024-09-17T04:33:13.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.913+0000 7f9a46e38640 1 -- 172.21.15.37:0/578420974 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9a24078b90 msgr2=0x7f9a2407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:13.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.913+0000 7f9a46e38640 1 --2- 172.21.15.37:0/578420974 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9a24078b90 0x7f9a2407b0a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c009230 tx=0x7f9a2c004730 comp rx=0 tx=0).stop 2024-09-17T04:33:13.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.913+0000 7f9a46e38640 1 -- 172.21.15.37:0/578420974 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9a400cd090 msgr2=0x7f9a40156210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:13.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.913+0000 7f9a46e38640 1 --2- 172.21.15.37:0/578420974 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9a400cd090 0x7f9a40156210 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400c760 tx=0x7f9a3400cc20 comp rx=0 tx=0).stop 2024-09-17T04:33:13.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.913+0000 7f9a46e38640 1 -- 172.21.15.37:0/578420974 shutdown_connections 2024-09-17T04:33:13.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.913+0000 7f9a46e38640 1 --2- 172.21.15.37:0/578420974 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9a24078b90 0x7f9a2407b0a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:13.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.913+0000 7f9a46e38640 1 --2- 172.21.15.37:0/578420974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a4015c830 0x7f9a40147ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:13.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.913+0000 7f9a46e38640 1 --2- 172.21.15.37:0/578420974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a400d1a10 0x7f9a40147580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:13.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.913+0000 7f9a46e38640 1 --2- 172.21.15.37:0/578420974 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9a400cd090 0x7f9a40156210 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:13.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.913+0000 7f9a46e38640 1 -- 172.21.15.37:0/578420974 >> 172.21.15.37:0/578420974 conn(0x7f9a40071050 msgr2=0x7f9a400b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:13.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.913+0000 7f9a46e38640 1 -- 172.21.15.37:0/578420974 shutdown_connections 2024-09-17T04:33:13.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:13.913+0000 7f9a46e38640 1 -- 172.21.15.37:0/578420974 wait complete. 2024-09-17T04:33:14.258 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:33:14.261 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:33:15.924 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:33:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.056+0000 7f1d56712640 1 -- 172.21.15.37:0/2809862299 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1d5015e830 msgr2=0x7f1d500b5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.056+0000 7f1d56712640 1 --2- 172.21.15.37:0/2809862299 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1d5015e830 0x7f1d500b5fc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1d40009830 tx=0x7f1d4002a510 comp rx=0 tx=0).stop 2024-09-17T04:33:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.056+0000 7f1d56712640 1 -- 172.21.15.37:0/2809862299 shutdown_connections 2024-09-17T04:33:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.056+0000 7f1d56712640 1 --2- 172.21.15.37:0/2809862299 >> v2:172.21.15.92:3300/0 conn(0x7f1d50162ec0 0x7f1d500d2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.056+0000 7f1d56712640 1 --2- 172.21.15.37:0/2809862299 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1d5015e830 0x7f1d500b5fc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.056+0000 7f1d56712640 1 -- 172.21.15.37:0/2809862299 >> 172.21.15.37:0/2809862299 conn(0x7f1d500d4540 msgr2=0x7f1d500d4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.056+0000 7f1d56712640 1 -- 172.21.15.37:0/2809862299 shutdown_connections 2024-09-17T04:33:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.056+0000 7f1d56712640 1 -- 172.21.15.37:0/2809862299 wait complete. 2024-09-17T04:33:16.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.058+0000 7f1d56712640 1 Processor -- start 2024-09-17T04:33:16.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.058+0000 7f1d56712640 1 -- start start 2024-09-17T04:33:16.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.059+0000 7f1d56712640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d500b6500 0x7f1d500cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:16.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.059+0000 7f1d56712640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d5015e830 0x7f1d50158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:16.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.059+0000 7f1d56712640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1d50162ec0 0x7f1d500cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:16.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.059+0000 7f1d56712640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1d500d30c0 con 0x7f1d500b6500 2024-09-17T04:33:16.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.059+0000 7f1d56712640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1d500d2f40 con 0x7f1d5015e830 2024-09-17T04:33:16.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.059+0000 7f1d56712640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f1d500d3240 con 0x7f1d50162ec0 2024-09-17T04:33:16.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.059+0000 7f1d4ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d500b6500 0x7f1d500cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:16.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.059+0000 7f1d4ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d500b6500 0x7f1d500cc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40522/0 (socket says 172.21.15.37:40522) 2024-09-17T04:33:16.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.059+0000 7f1d4ffff640 1 -- 172.21.15.37:0/1534869299 learned_addr learned my addr 172.21.15.37:0/1534869299 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.059+0000 7f1d54caf640 1 --2- 172.21.15.37:0/1534869299 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1d50162ec0 0x7f1d500cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.059+0000 7f1d47fff640 1 --2- 172.21.15.37:0/1534869299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d5015e830 0x7f1d50158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.059+0000 7f1d4ffff640 1 -- 172.21.15.37:0/1534869299 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1d50162ec0 msgr2=0x7f1d500cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.059+0000 7f1d4ffff640 1 --2- 172.21.15.37:0/1534869299 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1d50162ec0 0x7f1d500cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.059+0000 7f1d4ffff640 1 -- 172.21.15.37:0/1534869299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d5015e830 msgr2=0x7f1d50158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.059+0000 7f1d4ffff640 1 --2- 172.21.15.37:0/1534869299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d5015e830 0x7f1d50158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.059+0000 7f1d4ffff640 1 -- 172.21.15.37:0/1534869299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d50156c20 con 0x7f1d500b6500 2024-09-17T04:33:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.060+0000 7f1d4ffff640 1 --2- 172.21.15.37:0/1534869299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d500b6500 0x7f1d500cc120 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1d40009950 tx=0x7f1d40033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.060+0000 7f1d4d7fa640 1 -- 172.21.15.37:0/1534869299 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d40005c90 con 0x7f1d500b6500 2024-09-17T04:33:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.060+0000 7f1d4d7fa640 1 -- 172.21.15.37:0/1534869299 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d40031e50 con 0x7f1d500b6500 2024-09-17T04:33:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.060+0000 7f1d56712640 1 -- 172.21.15.37:0/1534869299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d500cad80 con 0x7f1d500b6500 2024-09-17T04:33:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.060+0000 7f1d4d7fa640 1 -- 172.21.15.37:0/1534869299 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d40005260 con 0x7f1d500b6500 2024-09-17T04:33:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.060+0000 7f1d56712640 1 -- 172.21.15.37:0/1534869299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d500c8340 con 0x7f1d500b6500 2024-09-17T04:33:16.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.061+0000 7f1d56712640 1 -- 172.21.15.37:0/1534869299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d500c8a60 con 0x7f1d500b6500 2024-09-17T04:33:16.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.062+0000 7f1d4d7fa640 1 -- 172.21.15.37:0/1534869299 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1d40005400 con 0x7f1d500b6500 2024-09-17T04:33:16.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.062+0000 7f1d4d7fa640 1 --2- 172.21.15.37:0/1534869299 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1d30078b90 0x7f1d3007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:16.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.063+0000 7f1d47fff640 1 --2- 172.21.15.37:0/1534869299 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1d30078b90 0x7f1d3007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:16.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.063+0000 7f1d4d7fa640 1 -- 172.21.15.37:0/1534869299 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f1d400bb8e0 con 0x7f1d500b6500 2024-09-17T04:33:16.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.063+0000 7f1d47fff640 1 --2- 172.21.15.37:0/1534869299 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1d30078b90 0x7f1d3007b0a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1d38007d60 tx=0x7f1d38007cf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:16.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.067+0000 7f1d4d7fa640 1 -- 172.21.15.37:0/1534869299 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1d4003f030 con 0x7f1d500b6500 2024-09-17T04:33:16.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.294+0000 7f1d56712640 1 -- 172.21.15.37:0/1534869299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d500d3760 con 0x7f1d500b6500 2024-09-17T04:33:16.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.294+0000 7f1d4d7fa640 1 -- 172.21.15.37:0/1534869299 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3906 (secure 0 0 0) 0x7f1d40083b40 con 0x7f1d500b6500 2024-09-17T04:33:16.294 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:16.294 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":3,"btime":"2024-09-17T04:33:14:643045+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4387,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/4285801054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4285801054},{"type":"v1","addr":"172.21.15.92:6836","nonce":4285801054}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4401,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/3514084301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3514084301},{"type":"v1","addr":"172.21.15.37:6834","nonce":3514084301}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4404,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3229527106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":3229527106},{"type":"v1","addr":"172.21.15.37:6835","nonce":3229527106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4419,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2128554253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2128554253},{"type":"v1","addr":"172.21.15.92:6837","nonce":2128554253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-17T04:33:16.295 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 3 2024-09-17T04:33:16.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.296+0000 7f1d56712640 1 -- 172.21.15.37:0/1534869299 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1d30078b90 msgr2=0x7f1d3007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:16.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.296+0000 7f1d56712640 1 --2- 172.21.15.37:0/1534869299 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1d30078b90 0x7f1d3007b0a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1d38007d60 tx=0x7f1d38007cf0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.296+0000 7f1d56712640 1 -- 172.21.15.37:0/1534869299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d500b6500 msgr2=0x7f1d500cc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:16.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.296+0000 7f1d56712640 1 --2- 172.21.15.37:0/1534869299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d500b6500 0x7f1d500cc120 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1d40009950 tx=0x7f1d40033c20 comp rx=0 tx=0).stop 2024-09-17T04:33:16.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.296+0000 7f1d56712640 1 -- 172.21.15.37:0/1534869299 shutdown_connections 2024-09-17T04:33:16.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.296+0000 7f1d56712640 1 --2- 172.21.15.37:0/1534869299 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1d30078b90 0x7f1d3007b0a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.297+0000 7f1d56712640 1 --2- 172.21.15.37:0/1534869299 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1d50162ec0 0x7f1d500cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.297+0000 7f1d56712640 1 --2- 172.21.15.37:0/1534869299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d5015e830 0x7f1d50158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.297+0000 7f1d56712640 1 --2- 172.21.15.37:0/1534869299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d500b6500 0x7f1d500cc120 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.297+0000 7f1d56712640 1 -- 172.21.15.37:0/1534869299 >> 172.21.15.37:0/1534869299 conn(0x7f1d500d4540 msgr2=0x7f1d50161080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:16.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.297+0000 7f1d56712640 1 -- 172.21.15.37:0/1534869299 shutdown_connections 2024-09-17T04:33:16.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.297+0000 7f1d56712640 1 -- 172.21.15.37:0/1534869299 wait complete. 2024-09-17T04:33:16.308 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T04:33:16.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.424+0000 7f54d547e640 1 -- 172.21.15.37:0/1221499098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54d00b6500 msgr2=0x7f54d01627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:16.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.424+0000 7f54d547e640 1 --2- 172.21.15.37:0/1221499098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54d00b6500 0x7f54d01627b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f54d00b6e10 tx=0x7f54c402a560 comp rx=0 tx=0).stop 2024-09-17T04:33:16.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.424+0000 7f54d547e640 1 -- 172.21.15.37:0/1221499098 shutdown_connections 2024-09-17T04:33:16.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.424+0000 7f54d547e640 1 --2- 172.21.15.37:0/1221499098 >> v2:172.21.15.92:3300/0 conn(0x7f54d0162cf0 0x7f54d006dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.424+0000 7f54d547e640 1 --2- 172.21.15.37:0/1221499098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54d00b6500 0x7f54d01627b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.424+0000 7f54d547e640 1 --2- 172.21.15.37:0/1221499098 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f54d015e830 0x7f54d00b5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.424+0000 7f54d547e640 1 -- 172.21.15.37:0/1221499098 >> 172.21.15.37:0/1221499098 conn(0x7f54d00d4540 msgr2=0x7f54d00d4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:16.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.424+0000 7f54d547e640 1 -- 172.21.15.37:0/1221499098 shutdown_connections 2024-09-17T04:33:16.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.424+0000 7f54d547e640 1 -- 172.21.15.37:0/1221499098 wait complete. 2024-09-17T04:33:16.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.426+0000 7f54d547e640 1 Processor -- start 2024-09-17T04:33:16.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.426+0000 7f54d547e640 1 -- start start 2024-09-17T04:33:16.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.427+0000 7f54d547e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f54d00b6500 0x7f54d00ce250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:16.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.427+0000 7f54d547e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54d015e830 0x7f54d015c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:16.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.427+0000 7f54d547e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54d0162cf0 0x7f54d015c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:16.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.427+0000 7f54d547e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f54d006e630 con 0x7f54d015e830 2024-09-17T04:33:16.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.427+0000 7f54d547e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f54d006e4b0 con 0x7f54d0162cf0 2024-09-17T04:33:16.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.427+0000 7f54d547e640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f54d006e7b0 con 0x7f54d00b6500 2024-09-17T04:33:16.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.428+0000 7f54ce7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54d015e830 0x7f54d015c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:16.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.428+0000 7f54ce7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54d015e830 0x7f54d015c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40540/0 (socket says 172.21.15.37:40540) 2024-09-17T04:33:16.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.428+0000 7f54ce7fc640 1 -- 172.21.15.37:0/3187242058 learned_addr learned my addr 172.21.15.37:0/3187242058 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:16.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.428+0000 7f54ceffd640 1 --2- 172.21.15.37:0/3187242058 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f54d00b6500 0x7f54d00ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:16.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.428+0000 7f54cf7fe640 1 --2- 172.21.15.37:0/3187242058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54d0162cf0 0x7f54d015c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:16.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.428+0000 7f54ce7fc640 1 -- 172.21.15.37:0/3187242058 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f54d00b6500 msgr2=0x7f54d00ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:16.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.428+0000 7f54ce7fc640 1 --2- 172.21.15.37:0/3187242058 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f54d00b6500 0x7f54d00ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.428+0000 7f54ce7fc640 1 -- 172.21.15.37:0/3187242058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54d0162cf0 msgr2=0x7f54d015c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:16.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.428+0000 7f54ce7fc640 1 --2- 172.21.15.37:0/3187242058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54d0162cf0 0x7f54d015c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.428+0000 7f54ce7fc640 1 -- 172.21.15.37:0/3187242058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d00cc660 con 0x7f54d015e830 2024-09-17T04:33:16.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.428+0000 7f54ce7fc640 1 --2- 172.21.15.37:0/3187242058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54d015e830 0x7f54d015c240 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f54c4034040 tx=0x7f54c40038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:16.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.428+0000 7f54b77fe640 1 -- 172.21.15.37:0/3187242058 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c403e030 con 0x7f54d015e830 2024-09-17T04:33:16.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.428+0000 7f54d547e640 1 -- 172.21.15.37:0/3187242058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d01f5500 con 0x7f54d015e830 2024-09-17T04:33:16.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.428+0000 7f54b77fe640 1 -- 172.21.15.37:0/3187242058 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54c4003ca0 con 0x7f54d015e830 2024-09-17T04:33:16.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.428+0000 7f54b77fe640 1 -- 172.21.15.37:0/3187242058 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4035040 con 0x7f54d015e830 2024-09-17T04:33:16.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.428+0000 7f54d547e640 1 -- 172.21.15.37:0/3187242058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d01f5ac0 con 0x7f54d015e830 2024-09-17T04:33:16.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.430+0000 7f54d547e640 1 -- 172.21.15.37:0/3187242058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54d00cc3d0 con 0x7f54d015e830 2024-09-17T04:33:16.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.430+0000 7f54b77fe640 1 -- 172.21.15.37:0/3187242058 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f54c4041030 con 0x7f54d015e830 2024-09-17T04:33:16.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.431+0000 7f54b77fe640 1 --2- 172.21.15.37:0/3187242058 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f54bc078b90 0x7f54bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:16.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.431+0000 7f54b77fe640 1 -- 172.21.15.37:0/3187242058 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f54c40bb020 con 0x7f54d015e830 2024-09-17T04:33:16.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.431+0000 7f54ceffd640 1 --2- 172.21.15.37:0/3187242058 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f54bc078b90 0x7f54bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:16.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.431+0000 7f54ceffd640 1 --2- 172.21.15.37:0/3187242058 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f54bc078b90 0x7f54bc07b0a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f54c0009ab0 tx=0x7f54c0005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:16.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.435+0000 7f54b77fe640 1 -- 172.21.15.37:0/3187242058 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f54c40830e0 con 0x7f54d015e830 2024-09-17T04:33:16.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.623+0000 7f54d547e640 1 -- 172.21.15.37:0/3187242058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f54d00b5fc0 con 0x7f54d015e830 2024-09-17T04:33:16.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.624+0000 7f54b77fe640 1 -- 172.21.15.37:0/3187242058 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) ==== 84+0+65 (secure 0 0 0) 0x7f54c4087b10 con 0x7f54d015e830 2024-09-17T04:33:16.624 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:16.624 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-09-17T04:33:16.624 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T04:33:16.624 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-09-17T04:33:16.624 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-09-17T04:33:16.624 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:16.624 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-09-17T04:33:16.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.626+0000 7f54d547e640 1 -- 172.21.15.37:0/3187242058 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f54bc078b90 msgr2=0x7f54bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:16.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.626+0000 7f54d547e640 1 --2- 172.21.15.37:0/3187242058 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f54bc078b90 0x7f54bc07b0a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f54c0009ab0 tx=0x7f54c0005f90 comp rx=0 tx=0).stop 2024-09-17T04:33:16.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.626+0000 7f54d547e640 1 -- 172.21.15.37:0/3187242058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54d015e830 msgr2=0x7f54d015c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:16.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.626+0000 7f54d547e640 1 --2- 172.21.15.37:0/3187242058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54d015e830 0x7f54d015c240 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f54c4034040 tx=0x7f54c40038a0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.626+0000 7f54d547e640 1 -- 172.21.15.37:0/3187242058 shutdown_connections 2024-09-17T04:33:16.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.626+0000 7f54d547e640 1 --2- 172.21.15.37:0/3187242058 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f54bc078b90 0x7f54bc07b0a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.626+0000 7f54d547e640 1 --2- 172.21.15.37:0/3187242058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54d0162cf0 0x7f54d015c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.626+0000 7f54d547e640 1 --2- 172.21.15.37:0/3187242058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f54d015e830 0x7f54d015c240 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.626+0000 7f54d547e640 1 --2- 172.21.15.37:0/3187242058 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f54d00b6500 0x7f54d00ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.626+0000 7f54d547e640 1 -- 172.21.15.37:0/3187242058 >> 172.21.15.37:0/3187242058 conn(0x7f54d00d4540 msgr2=0x7f54d00b9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.626+0000 7f54d547e640 1 -- 172.21.15.37:0/3187242058 shutdown_connections 2024-09-17T04:33:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.626+0000 7f54d547e640 1 -- 172.21.15.37:0/3187242058 wait complete. 2024-09-17T04:33:16.637 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T04:33:16.637 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T04:33:16.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.758+0000 7f2432d20640 1 -- 172.21.15.37:0/3895378692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242c0d1af0 msgr2=0x7f242c0b9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:16.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.758+0000 7f2432d20640 1 --2- 172.21.15.37:0/3895378692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242c0d1af0 0x7f242c0b9630 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f241c0098e0 tx=0x7f241c02a530 comp rx=0 tx=0).stop 2024-09-17T04:33:16.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.758+0000 7f2432d20640 1 -- 172.21.15.37:0/3895378692 shutdown_connections 2024-09-17T04:33:16.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.758+0000 7f2432d20640 1 --2- 172.21.15.37:0/3895378692 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f242c0d2c00 0x7f242c0d3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.758+0000 7f2432d20640 1 --2- 172.21.15.37:0/3895378692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242c0d1af0 0x7f242c0b9630 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.758+0000 7f2432d20640 1 -- 172.21.15.37:0/3895378692 >> 172.21.15.37:0/3895378692 conn(0x7f242c09d6f0 msgr2=0x7f242c0d4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:16.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.758+0000 7f2432d20640 1 -- 172.21.15.37:0/3895378692 shutdown_connections 2024-09-17T04:33:16.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.758+0000 7f2432d20640 1 -- 172.21.15.37:0/3895378692 wait complete. 2024-09-17T04:33:16.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.760+0000 7f2432d20640 1 Processor -- start 2024-09-17T04:33:16.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.760+0000 7f2432d20640 1 -- start start 2024-09-17T04:33:16.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.761+0000 7f2432d20640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f242c0b3b10 0x7f242c1f9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:16.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.761+0000 7f2432d20640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f242c0d1af0 0x7f242c1f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:16.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.761+0000 7f2432d20640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242c0d2c00 0x7f242c0ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:16.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.761+0000 7f2432d20640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f242c0d11e0 con 0x7f242c0d2c00 2024-09-17T04:33:16.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.761+0000 7f2432d20640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f242c0d1060 con 0x7f242c0b3b10 2024-09-17T04:33:16.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.761+0000 7f2432d20640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f242c0d1360 con 0x7f242c0d1af0 2024-09-17T04:33:16.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.761+0000 7f24312bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242c0d2c00 0x7f242c0ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:16.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.762+0000 7f24312bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242c0d2c00 0x7f242c0ccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40564/0 (socket says 172.21.15.37:40564) 2024-09-17T04:33:16.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.762+0000 7f24312bd640 1 -- 172.21.15.37:0/2805144660 learned_addr learned my addr 172.21.15.37:0/2805144660 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:16.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.762+0000 7f242bfff640 1 --2- 172.21.15.37:0/2805144660 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f242c0d1af0 0x7f242c1f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:16.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.762+0000 7f2430abc640 1 --2- 172.21.15.37:0/2805144660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f242c0b3b10 0x7f242c1f9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:16.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.762+0000 7f24312bd640 1 -- 172.21.15.37:0/2805144660 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f242c0d1af0 msgr2=0x7f242c1f6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:16.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.762+0000 7f24312bd640 1 --2- 172.21.15.37:0/2805144660 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f242c0d1af0 0x7f242c1f6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.762+0000 7f24312bd640 1 -- 172.21.15.37:0/2805144660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f242c0b3b10 msgr2=0x7f242c1f9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:16.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.762+0000 7f24312bd640 1 --2- 172.21.15.37:0/2805144660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f242c0b3b10 0x7f242c1f9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:16.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.762+0000 7f24312bd640 1 -- 172.21.15.37:0/2805144660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242c0cd310 con 0x7f242c0d2c00 2024-09-17T04:33:16.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.762+0000 7f24312bd640 1 --2- 172.21.15.37:0/2805144660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242c0d2c00 0x7f242c0ccdd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f242400c6e0 tx=0x7f242400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:16.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.762+0000 7f24297fa640 1 -- 172.21.15.37:0/2805144660 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2424003b70 con 0x7f242c0d2c00 2024-09-17T04:33:16.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.762+0000 7f24297fa640 1 -- 172.21.15.37:0/2805144660 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2424003d10 con 0x7f242c0d2c00 2024-09-17T04:33:16.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.762+0000 7f2432d20640 1 -- 172.21.15.37:0/2805144660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f242c0cd5e0 con 0x7f242c0d2c00 2024-09-17T04:33:16.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.762+0000 7f24297fa640 1 -- 172.21.15.37:0/2805144660 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2424005330 con 0x7f242c0d2c00 2024-09-17T04:33:16.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.762+0000 7f2432d20640 1 -- 172.21.15.37:0/2805144660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f242c1f5940 con 0x7f242c0d2c00 2024-09-17T04:33:16.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.763+0000 7f2432d20640 1 -- 172.21.15.37:0/2805144660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f242c0b9630 con 0x7f242c0d2c00 2024-09-17T04:33:16.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.764+0000 7f24297fa640 1 -- 172.21.15.37:0/2805144660 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f24240054d0 con 0x7f242c0d2c00 2024-09-17T04:33:16.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.764+0000 7f24297fa640 1 --2- 172.21.15.37:0/2805144660 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f2418078b90 0x7f241807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:16.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.764+0000 7f24297fa640 1 -- 172.21.15.37:0/2805144660 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f242409a5b0 con 0x7f242c0d2c00 2024-09-17T04:33:16.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.765+0000 7f2430abc640 1 --2- 172.21.15.37:0/2805144660 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f2418078b90 0x7f241807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:16.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.765+0000 7f2430abc640 1 --2- 172.21.15.37:0/2805144660 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f2418078b90 0x7f241807b0a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f241c034000 tx=0x7f241c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:16.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.769+0000 7f24297fa640 1 -- 172.21.15.37:0/2805144660 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f2424062790 con 0x7f242c0d2c00 2024-09-17T04:33:16.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:16.954+0000 7f2432d20640 1 -- 172.21.15.37:0/2805144660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f242c06d320 con 0x7f242c0d2c00 2024-09-17T04:33:17.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.657+0000 7f24297fa640 1 -- 172.21.15.37:0/2805144660 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) ==== 138+0+0 (secure 0 0 0) 0x7f24240671c0 con 0x7f242c0d2c00 2024-09-17T04:33:17.658 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-09-17T04:33:17.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.661+0000 7f2432d20640 1 -- 172.21.15.37:0/2805144660 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f2418078b90 msgr2=0x7f241807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:17.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.661+0000 7f2432d20640 1 --2- 172.21.15.37:0/2805144660 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f2418078b90 0x7f241807b0a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f241c034000 tx=0x7f241c004730 comp rx=0 tx=0).stop 2024-09-17T04:33:17.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.661+0000 7f2432d20640 1 -- 172.21.15.37:0/2805144660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242c0d2c00 msgr2=0x7f242c0ccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:17.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.661+0000 7f2432d20640 1 --2- 172.21.15.37:0/2805144660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242c0d2c00 0x7f242c0ccdd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f242400c6e0 tx=0x7f242400cba0 comp rx=0 tx=0).stop 2024-09-17T04:33:17.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.662+0000 7f2432d20640 1 -- 172.21.15.37:0/2805144660 shutdown_connections 2024-09-17T04:33:17.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.662+0000 7f2432d20640 1 --2- 172.21.15.37:0/2805144660 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f2418078b90 0x7f241807b0a0 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f241c034000 tx=0x7f241c004730 comp rx=0 tx=0).stop 2024-09-17T04:33:17.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.662+0000 7f2432d20640 1 --2- 172.21.15.37:0/2805144660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242c0d2c00 0x7f242c0ccdd0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:17.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.662+0000 7f2432d20640 1 --2- 172.21.15.37:0/2805144660 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f242c0d1af0 0x7f242c1f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:17.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.662+0000 7f2432d20640 1 --2- 172.21.15.37:0/2805144660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f242c0b3b10 0x7f242c1f9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:17.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.662+0000 7f2432d20640 1 -- 172.21.15.37:0/2805144660 >> 172.21.15.37:0/2805144660 conn(0x7f242c09d6f0 msgr2=0x7f242c1e6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:17.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.662+0000 7f2432d20640 1 -- 172.21.15.37:0/2805144660 shutdown_connections 2024-09-17T04:33:17.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.662+0000 7f2432d20640 1 -- 172.21.15.37:0/2805144660 wait complete. 2024-09-17T04:33:17.680 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T04:33:17.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.811+0000 7ff786e69640 1 -- 172.21.15.37:0/2406346967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff770003ae0 msgr2=0x7ff770003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:17.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.811+0000 7ff786e69640 1 --2- 172.21.15.37:0/2406346967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff770003ae0 0x7ff770003f50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff77400c2e0 tx=0x7ff770005830 comp rx=0 tx=0).stop 2024-09-17T04:33:17.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.811+0000 7ff785406640 1 -- 172.21.15.37:0/2406346967 reap_dead start 2024-09-17T04:33:17.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.811+0000 7ff786e69640 1 -- 172.21.15.37:0/2406346967 shutdown_connections 2024-09-17T04:33:17.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.811+0000 7ff786e69640 1 -- 172.21.15.37:0/2406346967 >> 172.21.15.37:0/2406346967 conn(0x7ff78009d7c0 msgr2=0x7ff7800d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:17.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.811+0000 7ff786e69640 1 -- 172.21.15.37:0/2406346967 shutdown_connections 2024-09-17T04:33:17.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.811+0000 7ff786e69640 1 -- 172.21.15.37:0/2406346967 wait complete. 2024-09-17T04:33:17.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.813+0000 7ff786e69640 1 Processor -- start 2024-09-17T04:33:17.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.813+0000 7ff786e69640 1 -- start start 2024-09-17T04:33:17.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.814+0000 7ff786e69640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff770003ae0 0x7ff7801f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:17.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.814+0000 7ff786e69640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7801f8980 0x7ff78015cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:17.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.814+0000 7ff786e69640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff7800c80d0 0x7ff7800cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:17.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.815+0000 7ff786e69640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff7800b1a80 con 0x7ff770003ae0 2024-09-17T04:33:17.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.815+0000 7ff786e69640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff7800b1900 con 0x7ff7801f8980 2024-09-17T04:33:17.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.815+0000 7ff786e69640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7ff780162640 con 0x7ff7800c80d0 2024-09-17T04:33:17.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.815+0000 7ff784c05640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff770003ae0 0x7ff7801f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:17.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.815+0000 7ff784c05640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff770003ae0 0x7ff7801f52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40584/0 (socket says 172.21.15.37:40584) 2024-09-17T04:33:17.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.815+0000 7ff784c05640 1 -- 172.21.15.37:0/570029397 learned_addr learned my addr 172.21.15.37:0/570029397 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:17.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.815+0000 7ff785406640 1 --2- 172.21.15.37:0/570029397 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff7800c80d0 0x7ff7800cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:17.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.815+0000 7ff77ffff640 1 --2- 172.21.15.37:0/570029397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7801f8980 0x7ff78015cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:17.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.815+0000 7ff784c05640 1 -- 172.21.15.37:0/570029397 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff7800c80d0 msgr2=0x7ff7800cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:17.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.815+0000 7ff784c05640 1 --2- 172.21.15.37:0/570029397 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff7800c80d0 0x7ff7800cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:17.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.815+0000 7ff784c05640 1 -- 172.21.15.37:0/570029397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7801f8980 msgr2=0x7ff78015cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:17.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.815+0000 7ff784c05640 1 --2- 172.21.15.37:0/570029397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7801f8980 0x7ff78015cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:17.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.815+0000 7ff784c05640 1 -- 172.21.15.37:0/570029397 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7801f5cc0 con 0x7ff770003ae0 2024-09-17T04:33:17.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.815+0000 7ff784c05640 1 --2- 172.21.15.37:0/570029397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff770003ae0 0x7ff7801f52e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff77402b130 tx=0x7ff774006ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:17.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.815+0000 7ff77d7fa640 1 -- 172.21.15.37:0/570029397 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff774031040 con 0x7ff770003ae0 2024-09-17T04:33:17.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.815+0000 7ff77d7fa640 1 -- 172.21.15.37:0/570029397 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff774003800 con 0x7ff770003ae0 2024-09-17T04:33:17.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.815+0000 7ff786e69640 1 -- 172.21.15.37:0/570029397 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7801f5fa0 con 0x7ff770003ae0 2024-09-17T04:33:17.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.815+0000 7ff77d7fa640 1 -- 172.21.15.37:0/570029397 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff774006600 con 0x7ff770003ae0 2024-09-17T04:33:17.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.816+0000 7ff786e69640 1 -- 172.21.15.37:0/570029397 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7800cba80 con 0x7ff770003ae0 2024-09-17T04:33:17.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.817+0000 7ff786e69640 1 -- 172.21.15.37:0/570029397 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7800b3210 con 0x7ff770003ae0 2024-09-17T04:33:17.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.817+0000 7ff77d7fa640 1 -- 172.21.15.37:0/570029397 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff774034070 con 0x7ff770003ae0 2024-09-17T04:33:17.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.818+0000 7ff77d7fa640 1 --2- 172.21.15.37:0/570029397 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff764078b90 0x7ff76407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:17.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.818+0000 7ff77d7fa640 1 -- 172.21.15.37:0/570029397 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5227+0+0 (secure 0 0 0) 0x7ff7740baf80 con 0x7ff770003ae0 2024-09-17T04:33:17.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.818+0000 7ff77ffff640 1 --2- 172.21.15.37:0/570029397 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff764078b90 0x7ff76407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:17.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.819+0000 7ff77ffff640 1 --2- 172.21.15.37:0/570029397 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff764078b90 0x7ff76407b0a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff76c005eb0 tx=0x7ff76c005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:17.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:17.823+0000 7ff77d7fa640 1 -- 172.21.15.37:0/570029397 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff774083610 con 0x7ff770003ae0 2024-09-17T04:33:18.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.009+0000 7ff786e69640 1 -- 172.21.15.37:0/570029397 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff7800b6e90 con 0x7ff770003ae0 2024-09-17T04:33:18.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.663+0000 7ff77d7fa640 1 -- 172.21.15.37:0/570029397 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) ==== 170+0+0 (secure 0 0 0) 0x7ff774082f10 con 0x7ff770003ae0 2024-09-17T04:33:18.663 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-09-17T04:33:18.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.666+0000 7ff786e69640 1 -- 172.21.15.37:0/570029397 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff764078b90 msgr2=0x7ff76407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:18.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.666+0000 7ff786e69640 1 --2- 172.21.15.37:0/570029397 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff764078b90 0x7ff76407b0a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff76c005eb0 tx=0x7ff76c005f90 comp rx=0 tx=0).stop 2024-09-17T04:33:18.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.666+0000 7ff786e69640 1 -- 172.21.15.37:0/570029397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff770003ae0 msgr2=0x7ff7801f52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:18.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.666+0000 7ff786e69640 1 --2- 172.21.15.37:0/570029397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff770003ae0 0x7ff7801f52e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff77402b130 tx=0x7ff774006ce0 comp rx=0 tx=0).stop 2024-09-17T04:33:18.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.667+0000 7ff786e69640 1 -- 172.21.15.37:0/570029397 shutdown_connections 2024-09-17T04:33:18.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.667+0000 7ff786e69640 1 --2- 172.21.15.37:0/570029397 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff764078b90 0x7ff76407b0a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:18.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.667+0000 7ff786e69640 1 --2- 172.21.15.37:0/570029397 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff7800c80d0 0x7ff7800cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:18.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.667+0000 7ff786e69640 1 --2- 172.21.15.37:0/570029397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7801f8980 0x7ff78015cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:18.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.667+0000 7ff786e69640 1 --2- 172.21.15.37:0/570029397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff770003ae0 0x7ff7801f52e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:18.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.667+0000 7ff786e69640 1 -- 172.21.15.37:0/570029397 >> 172.21.15.37:0/570029397 conn(0x7ff78009d7c0 msgr2=0x7ff7800d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:18.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.667+0000 7ff786e69640 1 -- 172.21.15.37:0/570029397 shutdown_connections 2024-09-17T04:33:18.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.667+0000 7ff786e69640 1 -- 172.21.15.37:0/570029397 wait complete. 2024-09-17T04:33:18.690 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T04:33:18.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.816+0000 7ff13199d640 1 -- 172.21.15.37:0/3426735796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff12c0d1a60 msgr2=0x7ff12c0cf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.816+0000 7ff13199d640 1 --2- 172.21.15.37:0/3426735796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff12c0d1a60 0x7ff12c0cf3e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff124005fd0 tx=0x7ff12402aa90 comp rx=0 tx=0).stop 2024-09-17T04:33:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.816+0000 7ff13199d640 1 -- 172.21.15.37:0/3426735796 shutdown_connections 2024-09-17T04:33:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.816+0000 7ff13199d640 1 --2- 172.21.15.37:0/3426735796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff12c0d1a60 0x7ff12c0cf3e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.816+0000 7ff13199d640 1 --2- 172.21.15.37:0/3426735796 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff12c0d1400 0x7ff12c0ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.816+0000 7ff13199d640 1 -- 172.21.15.37:0/3426735796 >> 172.21.15.37:0/3426735796 conn(0x7ff12c09d550 msgr2=0x7ff12c0d4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.816+0000 7ff13199d640 1 -- 172.21.15.37:0/3426735796 shutdown_connections 2024-09-17T04:33:18.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.816+0000 7ff13199d640 1 -- 172.21.15.37:0/3426735796 wait complete. 2024-09-17T04:33:18.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.818+0000 7ff13199d640 1 Processor -- start 2024-09-17T04:33:18.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.818+0000 7ff13199d640 1 -- start start 2024-09-17T04:33:18.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.820+0000 7ff13199d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff12c0d1a60 0x7ff12c0ce270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:18.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.820+0000 7ff13199d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff12c1e79c0 0x7ff12c1d4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:18.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.820+0000 7ff13199d640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff12c0b1ef0 0x7ff12c0b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:18.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.820+0000 7ff13199d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c0cfaa0 con 0x7ff12c0d1a60 2024-09-17T04:33:18.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.820+0000 7ff13199d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c0cf920 con 0x7ff12c1e79c0 2024-09-17T04:33:18.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.820+0000 7ff13199d640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7ff12c0cfc20 con 0x7ff12c0b1ef0 2024-09-17T04:33:18.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.820+0000 7ff13099b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff12c0d1a60 0x7ff12c0ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:18.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.820+0000 7ff13099b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff12c0d1a60 0x7ff12c0ce270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40610/0 (socket says 172.21.15.37:40610) 2024-09-17T04:33:18.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.820+0000 7ff13099b640 1 -- 172.21.15.37:0/2033705001 learned_addr learned my addr 172.21.15.37:0/2033705001 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:18.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.820+0000 7ff12bfff640 1 --2- 172.21.15.37:0/2033705001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff12c1e79c0 0x7ff12c1d4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:18.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.820+0000 7ff13119c640 1 --2- 172.21.15.37:0/2033705001 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff12c0b1ef0 0x7ff12c0b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:18.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.820+0000 7ff13099b640 1 -- 172.21.15.37:0/2033705001 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff12c0b1ef0 msgr2=0x7ff12c0b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:18.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.820+0000 7ff13099b640 1 --2- 172.21.15.37:0/2033705001 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff12c0b1ef0 0x7ff12c0b2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:18.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.820+0000 7ff13099b640 1 -- 172.21.15.37:0/2033705001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff12c1e79c0 msgr2=0x7ff12c1d4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:18.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.820+0000 7ff13099b640 1 --2- 172.21.15.37:0/2033705001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff12c1e79c0 0x7ff12c1d4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:18.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.820+0000 7ff13099b640 1 -- 172.21.15.37:0/2033705001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12c0b2840 con 0x7ff12c0d1a60 2024-09-17T04:33:18.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.820+0000 7ff13099b640 1 --2- 172.21.15.37:0/2033705001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff12c0d1a60 0x7ff12c0ce270 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff11400c3a0 tx=0x7ff11400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:18.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.820+0000 7ff1297fa640 1 -- 172.21.15.37:0/2033705001 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff114017070 con 0x7ff12c0d1a60 2024-09-17T04:33:18.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.820+0000 7ff1297fa640 1 -- 172.21.15.37:0/2033705001 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1140053c0 con 0x7ff12c0d1a60 2024-09-17T04:33:18.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.820+0000 7ff13199d640 1 -- 172.21.15.37:0/2033705001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff12c0ce7b0 con 0x7ff12c0d1a60 2024-09-17T04:33:18.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.821+0000 7ff1297fa640 1 -- 172.21.15.37:0/2033705001 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff114003850 con 0x7ff12c0d1a60 2024-09-17T04:33:18.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.821+0000 7ff13199d640 1 -- 172.21.15.37:0/2033705001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff12c1e8320 con 0x7ff12c0d1a60 2024-09-17T04:33:18.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.821+0000 7ff13199d640 1 -- 172.21.15.37:0/2033705001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff12c0d5010 con 0x7ff12c0d1a60 2024-09-17T04:33:18.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.822+0000 7ff1297fa640 1 -- 172.21.15.37:0/2033705001 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff11401a070 con 0x7ff12c0d1a60 2024-09-17T04:33:18.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.823+0000 7ff1297fa640 1 --2- 172.21.15.37:0/2033705001 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff10c078bc0 0x7ff10c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:18.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.823+0000 7ff1297fa640 1 -- 172.21.15.37:0/2033705001 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5609+0+0 (secure 0 0 0) 0x7ff114099ef0 con 0x7ff12c0d1a60 2024-09-17T04:33:18.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.823+0000 7ff12bfff640 1 --2- 172.21.15.37:0/2033705001 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff10c078bc0 0x7ff10c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:18.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.824+0000 7ff12bfff640 1 --2- 172.21.15.37:0/2033705001 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff10c078bc0 0x7ff10c07b0d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff12c1f81a0 tx=0x7ff11c005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:18.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:18.828+0000 7ff1297fa640 1 -- 172.21.15.37:0/2033705001 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff12c0b2840 con 0x7ff12c0d1a60 2024-09-17T04:33:19.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.049+0000 7ff13199d640 1 -- 172.21.15.37:0/2033705001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff12c0d0600 con 0x7ff12c0d1a60 2024-09-17T04:33:19.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.665+0000 7ff1297fa640 1 -- 172.21.15.37:0/2033705001 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7ff114061dd0 con 0x7ff12c0d1a60 2024-09-17T04:33:19.665 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 2 and data pool 3 2024-09-17T04:33:19.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.668+0000 7ff13199d640 1 -- 172.21.15.37:0/2033705001 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff10c078bc0 msgr2=0x7ff10c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:19.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.668+0000 7ff13199d640 1 --2- 172.21.15.37:0/2033705001 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff10c078bc0 0x7ff10c07b0d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff12c1f81a0 tx=0x7ff11c005f90 comp rx=0 tx=0).stop 2024-09-17T04:33:19.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.668+0000 7ff13199d640 1 -- 172.21.15.37:0/2033705001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff12c0d1a60 msgr2=0x7ff12c0ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:19.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.668+0000 7ff13199d640 1 --2- 172.21.15.37:0/2033705001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff12c0d1a60 0x7ff12c0ce270 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff11400c3a0 tx=0x7ff11400c860 comp rx=0 tx=0).stop 2024-09-17T04:33:19.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.669+0000 7ff13199d640 1 -- 172.21.15.37:0/2033705001 shutdown_connections 2024-09-17T04:33:19.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.669+0000 7ff13199d640 1 --2- 172.21.15.37:0/2033705001 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff10c078bc0 0x7ff10c07b0d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:19.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.669+0000 7ff13199d640 1 --2- 172.21.15.37:0/2033705001 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff12c0b1ef0 0x7ff12c0b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:19.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.669+0000 7ff13199d640 1 --2- 172.21.15.37:0/2033705001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff12c1e79c0 0x7ff12c1d4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:19.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.669+0000 7ff13199d640 1 --2- 172.21.15.37:0/2033705001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff12c0d1a60 0x7ff12c0ce270 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:19.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.669+0000 7ff13199d640 1 -- 172.21.15.37:0/2033705001 >> 172.21.15.37:0/2033705001 conn(0x7ff12c09d550 msgr2=0x7ff12c06eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:19.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.669+0000 7ff13199d640 1 -- 172.21.15.37:0/2033705001 shutdown_connections 2024-09-17T04:33:19.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.669+0000 7ff13199d640 1 -- 172.21.15.37:0/2033705001 wait complete. 2024-09-17T04:33:19.705 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:33:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.826+0000 7f541468b640 1 -- 172.21.15.37:0/335781202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f540c0cfe10 msgr2=0x7f540c162590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.826+0000 7f541468b640 1 --2- 172.21.15.37:0/335781202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f540c0cfe10 0x7f540c162590 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f540c0b7770 tx=0x7f540802a4d0 comp rx=0 tx=0).stop 2024-09-17T04:33:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.826+0000 7f541468b640 1 -- 172.21.15.37:0/335781202 shutdown_connections 2024-09-17T04:33:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.826+0000 7f541468b640 1 --2- 172.21.15.37:0/335781202 >> v2:172.21.15.92:3300/0 conn(0x7f540c15eba0 0x7f540c0d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.826+0000 7f541468b640 1 --2- 172.21.15.37:0/335781202 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f540c15e790 0x7f540c0d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.826+0000 7f541468b640 1 --2- 172.21.15.37:0/335781202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f540c0cfe10 0x7f540c162590 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.826+0000 7f541468b640 1 -- 172.21.15.37:0/335781202 >> 172.21.15.37:0/335781202 conn(0x7f540c06ddb0 msgr2=0x7f540c06e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.826+0000 7f541468b640 1 -- 172.21.15.37:0/335781202 shutdown_connections 2024-09-17T04:33:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.827+0000 7f541468b640 1 -- 172.21.15.37:0/335781202 wait complete. 2024-09-17T04:33:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.828+0000 7f541468b640 1 Processor -- start 2024-09-17T04:33:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.828+0000 7f541468b640 1 -- start start 2024-09-17T04:33:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.830+0000 7f541468b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f540c0cfe10 0x7f540c156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.830+0000 7f541468b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f540c15e790 0x7f540c0ce190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.830+0000 7f541468b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f540c15eba0 0x7f540c1f8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.830+0000 7f541468b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f540c0b6720 con 0x7f540c15e790 2024-09-17T04:33:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.830+0000 7f541468b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f540c0b65a0 con 0x7f540c15eba0 2024-09-17T04:33:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.830+0000 7f541468b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f540c0b68a0 con 0x7f540c0cfe10 2024-09-17T04:33:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.830+0000 7f5411c26640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f540c15e790 0x7f540c0ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.830+0000 7f5411c26640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f540c15e790 0x7f540c0ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40628/0 (socket says 172.21.15.37:40628) 2024-09-17T04:33:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.830+0000 7f5411c26640 1 -- 172.21.15.37:0/641975372 learned_addr learned my addr 172.21.15.37:0/641975372 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.830+0000 7f5412c28640 1 --2- 172.21.15.37:0/641975372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f540c15eba0 0x7f540c1f8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.830+0000 7f5412427640 1 --2- 172.21.15.37:0/641975372 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f540c0cfe10 0x7f540c156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.830+0000 7f5411c26640 1 -- 172.21.15.37:0/641975372 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f540c0cfe10 msgr2=0x7f540c156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.830+0000 7f5411c26640 1 --2- 172.21.15.37:0/641975372 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f540c0cfe10 0x7f540c156f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.830+0000 7f5411c26640 1 -- 172.21.15.37:0/641975372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f540c15eba0 msgr2=0x7f540c1f8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.830+0000 7f5411c26640 1 --2- 172.21.15.37:0/641975372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f540c15eba0 0x7f540c1f8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.830+0000 7f5411c26640 1 -- 172.21.15.37:0/641975372 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540c1555f0 con 0x7f540c15e790 2024-09-17T04:33:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.830+0000 7f5411c26640 1 --2- 172.21.15.37:0/641975372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f540c15e790 0x7f540c0ce190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00d6d0 tx=0x7f53fc00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.831+0000 7f5402ffd640 1 -- 172.21.15.37:0/641975372 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc017070 con 0x7f540c15e790 2024-09-17T04:33:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.831+0000 7f541468b640 1 -- 172.21.15.37:0/641975372 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540c1558d0 con 0x7f540c15e790 2024-09-17T04:33:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.831+0000 7f5402ffd640 1 -- 172.21.15.37:0/641975372 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53fc009b10 con 0x7f540c15e790 2024-09-17T04:33:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.831+0000 7f5402ffd640 1 -- 172.21.15.37:0/641975372 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc010460 con 0x7f540c15e790 2024-09-17T04:33:19.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.831+0000 7f541468b640 1 -- 172.21.15.37:0/641975372 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f540c15a3e0 con 0x7f540c15e790 2024-09-17T04:33:19.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.832+0000 7f541468b640 1 -- 172.21.15.37:0/641975372 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f540c162590 con 0x7f540c15e790 2024-09-17T04:33:19.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.832+0000 7f5402ffd640 1 -- 172.21.15.37:0/641975372 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f53fc01a070 con 0x7f540c15e790 2024-09-17T04:33:19.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.833+0000 7f5402ffd640 1 --2- 172.21.15.37:0/641975372 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f53f0078b90 0x7f53f007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:19.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.833+0000 7f5402ffd640 1 -- 172.21.15.37:0/641975372 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f53fc09ae60 con 0x7f540c15e790 2024-09-17T04:33:19.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.833+0000 7f5412427640 1 --2- 172.21.15.37:0/641975372 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f53f0078b90 0x7f53f007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:19.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.834+0000 7f5412427640 1 --2- 172.21.15.37:0/641975372 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f53f0078b90 0x7f53f007b0a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5408033000 tx=0x7f5408004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:19.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:19.838+0000 7f5402ffd640 1 -- 172.21.15.37:0/641975372 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f53fc062c60 con 0x7f540c15e790 2024-09-17T04:33:20.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.025+0000 7f541468b640 1 -- 172.21.15.37:0/641975372 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f540c154050 con 0x7f540c15e790 2024-09-17T04:33:20.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.026+0000 7f5402ffd640 1 -- 172.21.15.37:0/641975372 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16618 (secure 0 0 0) 0x7f53fc067690 con 0x7f540c15e790 2024-09-17T04:33:20.026 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:20.026 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":21,"fsid":"a908a603-430c-4209-8294-0c3460b35f42","created":"2024-09-17T04:32:54.139221+0000","modified":"2024-09-17T04:33:19.050618+0000","last_up_change":"2024-09-17T04:33:06.922210+0000","last_in_change":"2024-09-17T04:32:57.754118+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:33:08.461907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:33:16.958785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:18.013768+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ea4b8d1-0d22-4dd9-b913-9df585ef24ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6809","nonce":2283587593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6811","nonce":2283587593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6815","nonce":2283587593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6813","nonce":2283587593}]},"public_addr":"172.21.15.37:6809/2283587593","cluster_addr":"172.21.15.37:6811/2283587593","heartbeat_back_addr":"172.21.15.37:6815/2283587593","heartbeat_front_addr":"172.21.15.37:6813/2283587593","state":["exists","up"]},{"osd":1,"uuid":"ac745a33-e8ad-4083-9c18-71bf81982b76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6825","nonce":2657955935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6827","nonce":2657955935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6831","nonce":2657955935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6829","nonce":2657955935}]},"public_addr":"172.21.15.37:6825/2657955935","cluster_addr":"172.21.15.37:6827/2657955935","heartbeat_back_addr":"172.21.15.37:6831/2657955935","heartbeat_front_addr":"172.21.15.37:6829/2657955935","state":["exists","up"]},{"osd":2,"uuid":"3c63f3d8-132a-4735-b0bc-d64313a46503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6817","nonce":579720176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6819","nonce":579720176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6823","nonce":579720176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6821","nonce":579720176}]},"public_addr":"172.21.15.37:6817/579720176","cluster_addr":"172.21.15.37:6819/579720176","heartbeat_back_addr":"172.21.15.37:6823/579720176","heartbeat_front_addr":"172.21.15.37:6821/579720176","state":["exists","up"]},{"osd":3,"uuid":"4f8c6ca2-ea1d-4610-9127-52c4dd8bc8e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6801","nonce":809975826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6803","nonce":809975826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6807","nonce":809975826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6805","nonce":809975826}]},"public_addr":"172.21.15.37:6801/809975826","cluster_addr":"172.21.15.37:6803/809975826","heartbeat_back_addr":"172.21.15.37:6807/809975826","heartbeat_front_addr":"172.21.15.37:6805/809975826","state":["exists","up"]},{"osd":4,"uuid":"63d0c4ab-f115-47b4-9611-12a10153473a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6819","nonce":2509785752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6821","nonce":2509785752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6825","nonce":2509785752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6823","nonce":2509785752}]},"public_addr":"172.21.15.92:6819/2509785752","cluster_addr":"172.21.15.92:6821/2509785752","heartbeat_back_addr":"172.21.15.92:6825/2509785752","heartbeat_front_addr":"172.21.15.92:6823/2509785752","state":["exists","up"]},{"osd":5,"uuid":"4c56c04d-0c07-4f76-9d75-6335121310ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6827","nonce":248263832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6829","nonce":248263832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6833","nonce":248263832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6831","nonce":248263832}]},"public_addr":"172.21.15.92:6827/248263832","cluster_addr":"172.21.15.92:6829/248263832","heartbeat_back_addr":"172.21.15.92:6833/248263832","heartbeat_front_addr":"172.21.15.92:6831/248263832","state":["exists","up"]},{"osd":6,"uuid":"b30eae7c-096a-4d24-ac51-96b8b13ecaee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6811","nonce":4078365985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6813","nonce":4078365985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6817","nonce":4078365985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6815","nonce":4078365985}]},"public_addr":"172.21.15.92:6811/4078365985","cluster_addr":"172.21.15.92:6813/4078365985","heartbeat_back_addr":"172.21.15.92:6817/4078365985","heartbeat_front_addr":"172.21.15.92:6815/4078365985","state":["exists","up"]},{"osd":7,"uuid":"62f12540-d374-4a4c-9ff7-63b7304fe77f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6803","nonce":3444787223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6805","nonce":3444787223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6809","nonce":3444787223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6807","nonce":3444787223}]},"public_addr":"172.21.15.92:6803/3444787223","cluster_addr":"172.21.15.92:6805/3444787223","heartbeat_back_addr":"172.21.15.92:6809/3444787223","heartbeat_front_addr":"172.21.15.92:6807/3444787223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.006420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.111135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.103325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.072656+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.200989+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.060455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.774773+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.865778+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:33:20.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.028+0000 7f541468b640 1 -- 172.21.15.37:0/641975372 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f53f0078b90 msgr2=0x7f53f007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:20.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.028+0000 7f541468b640 1 --2- 172.21.15.37:0/641975372 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f53f0078b90 0x7f53f007b0a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5408033000 tx=0x7f5408004730 comp rx=0 tx=0).stop 2024-09-17T04:33:20.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.028+0000 7f541468b640 1 -- 172.21.15.37:0/641975372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f540c15e790 msgr2=0x7f540c0ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:20.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.028+0000 7f541468b640 1 --2- 172.21.15.37:0/641975372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f540c15e790 0x7f540c0ce190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00d6d0 tx=0x7f53fc00db90 comp rx=0 tx=0).stop 2024-09-17T04:33:20.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.028+0000 7f541468b640 1 -- 172.21.15.37:0/641975372 shutdown_connections 2024-09-17T04:33:20.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.028+0000 7f541468b640 1 --2- 172.21.15.37:0/641975372 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f53f0078b90 0x7f53f007b0a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.028+0000 7f541468b640 1 --2- 172.21.15.37:0/641975372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f540c15eba0 0x7f540c1f8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.028+0000 7f541468b640 1 --2- 172.21.15.37:0/641975372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f540c15e790 0x7f540c0ce190 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.028+0000 7f541468b640 1 --2- 172.21.15.37:0/641975372 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f540c0cfe10 0x7f540c156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.028+0000 7f541468b640 1 -- 172.21.15.37:0/641975372 >> 172.21.15.37:0/641975372 conn(0x7f540c06ddb0 msgr2=0x7f540c0b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:20.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.028+0000 7f541468b640 1 -- 172.21.15.37:0/641975372 shutdown_connections 2024-09-17T04:33:20.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.028+0000 7f541468b640 1 -- 172.21.15.37:0/641975372 wait complete. 2024-09-17T04:33:20.039 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:33:20.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.156+0000 7fa7a26bd640 1 -- 172.21.15.37:0/4228561031 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa79c0d2f70 msgr2=0x7fa79c162510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:20.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.156+0000 7fa7a26bd640 1 --2- 172.21.15.37:0/4228561031 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa79c0d2f70 0x7fa79c162510 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa790009830 tx=0x7fa79002a3f0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.156+0000 7fa7a26bd640 1 -- 172.21.15.37:0/4228561031 shutdown_connections 2024-09-17T04:33:20.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.156+0000 7fa7a26bd640 1 --2- 172.21.15.37:0/4228561031 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa79c0d2f70 0x7fa79c162510 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.156+0000 7fa7a26bd640 1 --2- 172.21.15.37:0/4228561031 >> v2:172.21.15.92:3300/0 conn(0x7fa79c0b32c0 0x7fa79c0b6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.156+0000 7fa7a26bd640 1 -- 172.21.15.37:0/4228561031 >> 172.21.15.37:0/4228561031 conn(0x7fa79c09d790 msgr2=0x7fa79c0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:20.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.156+0000 7fa7a26bd640 1 -- 172.21.15.37:0/4228561031 shutdown_connections 2024-09-17T04:33:20.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.156+0000 7fa7a26bd640 1 -- 172.21.15.37:0/4228561031 wait complete. 2024-09-17T04:33:20.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.157+0000 7fa7a26bd640 1 Processor -- start 2024-09-17T04:33:20.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.158+0000 7fa7a26bd640 1 -- start start 2024-09-17T04:33:20.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.159+0000 7fa7a26bd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa79c0b32c0 0x7fa79c150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:20.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.159+0000 7fa7a26bd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa79c0d2f70 0x7fa79c1f8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:20.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.159+0000 7fa7a26bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa79c0d3380 0x7fa79c1f9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:20.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.159+0000 7fa7a26bd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c0b11b0 con 0x7fa79c0d3380 2024-09-17T04:33:20.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.159+0000 7fa7a26bd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c0b1030 con 0x7fa79c0d2f70 2024-09-17T04:33:20.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.159+0000 7fa7a26bd640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fa79c0b1330 con 0x7fa79c0b32c0 2024-09-17T04:33:20.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.159+0000 7fa7a0c5a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa79c0d3380 0x7fa79c1f9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:20.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.159+0000 7fa7a0c5a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa79c0d3380 0x7fa79c1f9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40630/0 (socket says 172.21.15.37:40630) 2024-09-17T04:33:20.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.159+0000 7fa7a0c5a640 1 -- 172.21.15.37:0/1406061291 learned_addr learned my addr 172.21.15.37:0/1406061291 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:20.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.159+0000 7fa79b7fe640 1 --2- 172.21.15.37:0/1406061291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa79c0d2f70 0x7fa79c1f8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:20.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.159+0000 7fa79bfff640 1 --2- 172.21.15.37:0/1406061291 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa79c0b32c0 0x7fa79c150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:20.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.159+0000 7fa7a0c5a640 1 -- 172.21.15.37:0/1406061291 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa79c0b32c0 msgr2=0x7fa79c150bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:20.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.159+0000 7fa7a0c5a640 1 --2- 172.21.15.37:0/1406061291 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa79c0b32c0 0x7fa79c150bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.159+0000 7fa7a0c5a640 1 -- 172.21.15.37:0/1406061291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa79c0d2f70 msgr2=0x7fa79c1f8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:20.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.159+0000 7fa7a0c5a640 1 --2- 172.21.15.37:0/1406061291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa79c0d2f70 0x7fa79c1f8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.159+0000 7fa7a0c5a640 1 -- 172.21.15.37:0/1406061291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa79c1515e0 con 0x7fa79c0d3380 2024-09-17T04:33:20.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.160+0000 7fa7a0c5a640 1 --2- 172.21.15.37:0/1406061291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa79c0d3380 0x7fa79c1f9300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa79400cae0 tx=0x7fa79400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:20.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.160+0000 7fa798ff9640 1 -- 172.21.15.37:0/1406061291 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa794016020 con 0x7fa79c0d3380 2024-09-17T04:33:20.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.160+0000 7fa798ff9640 1 -- 172.21.15.37:0/1406061291 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa79400acd0 con 0x7fa79c0d3380 2024-09-17T04:33:20.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.160+0000 7fa7a26bd640 1 -- 172.21.15.37:0/1406061291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa79c09e350 con 0x7fa79c0d3380 2024-09-17T04:33:20.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.160+0000 7fa798ff9640 1 -- 172.21.15.37:0/1406061291 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa794012680 con 0x7fa79c0d3380 2024-09-17T04:33:20.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.160+0000 7fa7a26bd640 1 -- 172.21.15.37:0/1406061291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa79c0b7330 con 0x7fa79c0d3380 2024-09-17T04:33:20.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.161+0000 7fa7a26bd640 1 -- 172.21.15.37:0/1406061291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa79c0b39c0 con 0x7fa79c0d3380 2024-09-17T04:33:20.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.162+0000 7fa798ff9640 1 -- 172.21.15.37:0/1406061291 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa794019020 con 0x7fa79c0d3380 2024-09-17T04:33:20.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.162+0000 7fa798ff9640 1 --2- 172.21.15.37:0/1406061291 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa788078bc0 0x7fa78807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:20.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.162+0000 7fa798ff9640 1 -- 172.21.15.37:0/1406061291 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa79409aec0 con 0x7fa79c0d3380 2024-09-17T04:33:20.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.163+0000 7fa79bfff640 1 --2- 172.21.15.37:0/1406061291 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa788078bc0 0x7fa78807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:20.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.163+0000 7fa79bfff640 1 --2- 172.21.15.37:0/1406061291 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa788078bc0 0x7fa78807b0d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0095f0 tx=0x7fa78c005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:20.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.167+0000 7fa798ff9640 1 -- 172.21.15.37:0/1406061291 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa79c0b39c0 con 0x7fa79c0d3380 2024-09-17T04:33:20.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.354+0000 7fa7a26bd640 1 -- 172.21.15.37:0/1406061291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa79c0b2010 con 0x7fa79c0d3380 2024-09-17T04:33:20.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.355+0000 7fa798ff9640 1 -- 172.21.15.37:0/1406061291 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16618 (secure 0 0 0) 0x7fa794020030 con 0x7fa79c0d3380 2024-09-17T04:33:20.355 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:20.355 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":21,"fsid":"a908a603-430c-4209-8294-0c3460b35f42","created":"2024-09-17T04:32:54.139221+0000","modified":"2024-09-17T04:33:19.050618+0000","last_up_change":"2024-09-17T04:33:06.922210+0000","last_in_change":"2024-09-17T04:32:57.754118+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:33:08.461907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:33:16.958785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:18.013768+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ea4b8d1-0d22-4dd9-b913-9df585ef24ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6809","nonce":2283587593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6811","nonce":2283587593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6815","nonce":2283587593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6813","nonce":2283587593}]},"public_addr":"172.21.15.37:6809/2283587593","cluster_addr":"172.21.15.37:6811/2283587593","heartbeat_back_addr":"172.21.15.37:6815/2283587593","heartbeat_front_addr":"172.21.15.37:6813/2283587593","state":["exists","up"]},{"osd":1,"uuid":"ac745a33-e8ad-4083-9c18-71bf81982b76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6825","nonce":2657955935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6827","nonce":2657955935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6831","nonce":2657955935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6829","nonce":2657955935}]},"public_addr":"172.21.15.37:6825/2657955935","cluster_addr":"172.21.15.37:6827/2657955935","heartbeat_back_addr":"172.21.15.37:6831/2657955935","heartbeat_front_addr":"172.21.15.37:6829/2657955935","state":["exists","up"]},{"osd":2,"uuid":"3c63f3d8-132a-4735-b0bc-d64313a46503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6817","nonce":579720176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6819","nonce":579720176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6823","nonce":579720176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6821","nonce":579720176}]},"public_addr":"172.21.15.37:6817/579720176","cluster_addr":"172.21.15.37:6819/579720176","heartbeat_back_addr":"172.21.15.37:6823/579720176","heartbeat_front_addr":"172.21.15.37:6821/579720176","state":["exists","up"]},{"osd":3,"uuid":"4f8c6ca2-ea1d-4610-9127-52c4dd8bc8e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6801","nonce":809975826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6803","nonce":809975826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6807","nonce":809975826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6805","nonce":809975826}]},"public_addr":"172.21.15.37:6801/809975826","cluster_addr":"172.21.15.37:6803/809975826","heartbeat_back_addr":"172.21.15.37:6807/809975826","heartbeat_front_addr":"172.21.15.37:6805/809975826","state":["exists","up"]},{"osd":4,"uuid":"63d0c4ab-f115-47b4-9611-12a10153473a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6819","nonce":2509785752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6821","nonce":2509785752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6825","nonce":2509785752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6823","nonce":2509785752}]},"public_addr":"172.21.15.92:6819/2509785752","cluster_addr":"172.21.15.92:6821/2509785752","heartbeat_back_addr":"172.21.15.92:6825/2509785752","heartbeat_front_addr":"172.21.15.92:6823/2509785752","state":["exists","up"]},{"osd":5,"uuid":"4c56c04d-0c07-4f76-9d75-6335121310ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6827","nonce":248263832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6829","nonce":248263832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6833","nonce":248263832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6831","nonce":248263832}]},"public_addr":"172.21.15.92:6827/248263832","cluster_addr":"172.21.15.92:6829/248263832","heartbeat_back_addr":"172.21.15.92:6833/248263832","heartbeat_front_addr":"172.21.15.92:6831/248263832","state":["exists","up"]},{"osd":6,"uuid":"b30eae7c-096a-4d24-ac51-96b8b13ecaee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6811","nonce":4078365985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6813","nonce":4078365985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6817","nonce":4078365985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6815","nonce":4078365985}]},"public_addr":"172.21.15.92:6811/4078365985","cluster_addr":"172.21.15.92:6813/4078365985","heartbeat_back_addr":"172.21.15.92:6817/4078365985","heartbeat_front_addr":"172.21.15.92:6815/4078365985","state":["exists","up"]},{"osd":7,"uuid":"62f12540-d374-4a4c-9ff7-63b7304fe77f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6803","nonce":3444787223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6805","nonce":3444787223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6809","nonce":3444787223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6807","nonce":3444787223}]},"public_addr":"172.21.15.92:6803/3444787223","cluster_addr":"172.21.15.92:6805/3444787223","heartbeat_back_addr":"172.21.15.92:6809/3444787223","heartbeat_front_addr":"172.21.15.92:6807/3444787223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.006420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.111135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.103325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.072656+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.200989+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.060455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.774773+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.865778+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:33:20.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.357+0000 7fa7a26bd640 1 -- 172.21.15.37:0/1406061291 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa788078bc0 msgr2=0x7fa78807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:20.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.357+0000 7fa7a26bd640 1 --2- 172.21.15.37:0/1406061291 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa788078bc0 0x7fa78807b0d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0095f0 tx=0x7fa78c005f90 comp rx=0 tx=0).stop 2024-09-17T04:33:20.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.357+0000 7fa7a26bd640 1 -- 172.21.15.37:0/1406061291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa79c0d3380 msgr2=0x7fa79c1f9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:20.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.357+0000 7fa7a26bd640 1 --2- 172.21.15.37:0/1406061291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa79c0d3380 0x7fa79c1f9300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa79400cae0 tx=0x7fa79400a060 comp rx=0 tx=0).stop 2024-09-17T04:33:20.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.357+0000 7fa7a26bd640 1 -- 172.21.15.37:0/1406061291 shutdown_connections 2024-09-17T04:33:20.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.357+0000 7fa7a26bd640 1 --2- 172.21.15.37:0/1406061291 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa788078bc0 0x7fa78807b0d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.357+0000 7fa7a26bd640 1 --2- 172.21.15.37:0/1406061291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa79c0d3380 0x7fa79c1f9300 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.357+0000 7fa7a26bd640 1 --2- 172.21.15.37:0/1406061291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa79c0d2f70 0x7fa79c1f8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.357+0000 7fa7a26bd640 1 --2- 172.21.15.37:0/1406061291 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa79c0b32c0 0x7fa79c150bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.357+0000 7fa7a26bd640 1 -- 172.21.15.37:0/1406061291 >> 172.21.15.37:0/1406061291 conn(0x7fa79c09d790 msgr2=0x7fa79c161ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:20.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.357+0000 7fa7a26bd640 1 -- 172.21.15.37:0/1406061291 shutdown_connections 2024-09-17T04:33:20.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.357+0000 7fa7a26bd640 1 -- 172.21.15.37:0/1406061291 wait complete. 2024-09-17T04:33:20.369 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T04:33:20.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.490+0000 7fb20255f640 1 -- 172.21.15.37:0/2748314214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1e4004800 msgr2=0x7fb1e4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:20.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.490+0000 7fb20255f640 1 --2- 172.21.15.37:0/2748314214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1e4004800 0x7fb1e4004c10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0006fc0 tx=0x7fb1e4005830 comp rx=0 tx=0).stop 2024-09-17T04:33:20.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.490+0000 7fb200afc640 1 -- 172.21.15.37:0/2748314214 reap_dead start 2024-09-17T04:33:20.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.490+0000 7fb20255f640 1 -- 172.21.15.37:0/2748314214 shutdown_connections 2024-09-17T04:33:20.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.490+0000 7fb20255f640 1 -- 172.21.15.37:0/2748314214 >> 172.21.15.37:0/2748314214 conn(0x7fb1fc09d790 msgr2=0x7fb1fc0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:20.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.490+0000 7fb20255f640 1 -- 172.21.15.37:0/2748314214 shutdown_connections 2024-09-17T04:33:20.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.490+0000 7fb20255f640 1 -- 172.21.15.37:0/2748314214 wait complete. 2024-09-17T04:33:20.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.492+0000 7fb20255f640 1 Processor -- start 2024-09-17T04:33:20.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.492+0000 7fb20255f640 1 -- start start 2024-09-17T04:33:20.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.493+0000 7fb20255f640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb1e4004800 0x7fb1fc0cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:20.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.493+0000 7fb20255f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1fc15ba10 0x7fb1fc15eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:20.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.494+0000 7fb20255f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1fc15f2e0 0x7fb1fc0b8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:20.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.494+0000 7fb20255f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc0b6a40 con 0x7fb1fc15ba10 2024-09-17T04:33:20.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.494+0000 7fb20255f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc0b68c0 con 0x7fb1fc15f2e0 2024-09-17T04:33:20.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.494+0000 7fb20255f640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fb1fc0b1030 con 0x7fb1e4004800 2024-09-17T04:33:20.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.494+0000 7fb1fb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1fc15ba10 0x7fb1fc15eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:20.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.494+0000 7fb1fb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1fc15ba10 0x7fb1fc15eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40650/0 (socket says 172.21.15.37:40650) 2024-09-17T04:33:20.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.494+0000 7fb1fb7fe640 1 -- 172.21.15.37:0/124747677 learned_addr learned my addr 172.21.15.37:0/124747677 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:20.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.494+0000 7fb1fbfff640 1 --2- 172.21.15.37:0/124747677 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb1e4004800 0x7fb1fc0cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:20.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.494+0000 7fb200afc640 1 --2- 172.21.15.37:0/124747677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1fc15f2e0 0x7fb1fc0b8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:20.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.494+0000 7fb1fb7fe640 1 -- 172.21.15.37:0/124747677 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb1e4004800 msgr2=0x7fb1fc0cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:20.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.494+0000 7fb1fb7fe640 1 --2- 172.21.15.37:0/124747677 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb1e4004800 0x7fb1fc0cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.494+0000 7fb1fb7fe640 1 -- 172.21.15.37:0/124747677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1fc15f2e0 msgr2=0x7fb1fc0b8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:20.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.494+0000 7fb1fb7fe640 1 --2- 172.21.15.37:0/124747677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1fc15f2e0 0x7fb1fc0b8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:20.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.494+0000 7fb1fb7fe640 1 -- 172.21.15.37:0/124747677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1fc1f8e60 con 0x7fb1fc15ba10 2024-09-17T04:33:20.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.494+0000 7fb1fb7fe640 1 --2- 172.21.15.37:0/124747677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1fc15ba10 0x7fb1fc15eda0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000c350 tx=0x7fb1f0001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:20.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.494+0000 7fb1f8ff9640 1 -- 172.21.15.37:0/124747677 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0031040 con 0x7fb1fc15ba10 2024-09-17T04:33:20.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.494+0000 7fb1f8ff9640 1 -- 172.21.15.37:0/124747677 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1f00035f0 con 0x7fb1fc15ba10 2024-09-17T04:33:20.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.494+0000 7fb20255f640 1 -- 172.21.15.37:0/124747677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1fc1f9140 con 0x7fb1fc15ba10 2024-09-17T04:33:20.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.494+0000 7fb1f8ff9640 1 -- 172.21.15.37:0/124747677 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0003a20 con 0x7fb1fc15ba10 2024-09-17T04:33:20.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.494+0000 7fb20255f640 1 -- 172.21.15.37:0/124747677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1fc1f9700 con 0x7fb1fc15ba10 2024-09-17T04:33:20.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.496+0000 7fb20255f640 1 -- 172.21.15.37:0/124747677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1fc1f5490 con 0x7fb1fc15ba10 2024-09-17T04:33:20.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.496+0000 7fb1f8ff9640 1 -- 172.21.15.37:0/124747677 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb1f0034070 con 0x7fb1fc15ba10 2024-09-17T04:33:20.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.497+0000 7fb1f8ff9640 1 --2- 172.21.15.37:0/124747677 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb1e8078b90 0x7fb1e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:20.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.497+0000 7fb1f8ff9640 1 -- 172.21.15.37:0/124747677 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb1f00baa40 con 0x7fb1fc15ba10 2024-09-17T04:33:20.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.497+0000 7fb1fbfff640 1 --2- 172.21.15.37:0/124747677 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb1e8078b90 0x7fb1e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:20.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.497+0000 7fb1fbfff640 1 --2- 172.21.15.37:0/124747677 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb1e8078b90 0x7fb1e807b0a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec000c00 tx=0x7fb1ec006b70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:20.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.501+0000 7fb1f8ff9640 1 -- 172.21.15.37:0/124747677 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb1f0082950 con 0x7fb1fc15ba10 2024-09-17T04:33:20.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:20.727+0000 7fb20255f640 1 -- 172.21.15.37:0/124747677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb1fc1f8bf0 con 0x7fb1fc15ba10 2024-09-17T04:33:21.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.693+0000 7fb1f8ff9640 1 -- 172.21.15.37:0/124747677 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7fb1f00bae20 con 0x7fb1fc15ba10 2024-09-17T04:33:21.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.696+0000 7fb20255f640 1 -- 172.21.15.37:0/124747677 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb1e8078b90 msgr2=0x7fb1e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:21.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.696+0000 7fb20255f640 1 --2- 172.21.15.37:0/124747677 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb1e8078b90 0x7fb1e807b0a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec000c00 tx=0x7fb1ec006b70 comp rx=0 tx=0).stop 2024-09-17T04:33:21.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.696+0000 7fb20255f640 1 -- 172.21.15.37:0/124747677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1fc15ba10 msgr2=0x7fb1fc15eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:21.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.696+0000 7fb20255f640 1 --2- 172.21.15.37:0/124747677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1fc15ba10 0x7fb1fc15eda0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000c350 tx=0x7fb1f0001cb0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.696+0000 7fb20255f640 1 -- 172.21.15.37:0/124747677 shutdown_connections 2024-09-17T04:33:21.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.696+0000 7fb20255f640 1 --2- 172.21.15.37:0/124747677 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb1e8078b90 0x7fb1e807b0a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.696+0000 7fb20255f640 1 --2- 172.21.15.37:0/124747677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1fc15f2e0 0x7fb1fc0b8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.696+0000 7fb20255f640 1 --2- 172.21.15.37:0/124747677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1fc15ba10 0x7fb1fc15eda0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.696+0000 7fb20255f640 1 --2- 172.21.15.37:0/124747677 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb1e4004800 0x7fb1fc0cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.696+0000 7fb20255f640 1 -- 172.21.15.37:0/124747677 >> 172.21.15.37:0/124747677 conn(0x7fb1fc09d790 msgr2=0x7fb1fc0d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:21.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.696+0000 7fb20255f640 1 -- 172.21.15.37:0/124747677 shutdown_connections 2024-09-17T04:33:21.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.696+0000 7fb20255f640 1 -- 172.21.15.37:0/124747677 wait complete. 2024-09-17T04:33:21.709 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T04:33:21.709 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T04:33:21.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.837+0000 7f43af769640 1 -- 172.21.15.37:0/2289235496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4394003d10 msgr2=0x7f4394004160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:21.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.837+0000 7f43af769640 1 --2- 172.21.15.37:0/2289235496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4394003d10 0x7f4394004160 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f43a4001830 tx=0x7f4394005830 comp rx=0 tx=0).stop 2024-09-17T04:33:21.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.837+0000 7f43af769640 1 -- 172.21.15.37:0/2289235496 shutdown_connections 2024-09-17T04:33:21.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.837+0000 7f43af769640 1 --2- 172.21.15.37:0/2289235496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4394003d10 0x7f4394004160 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.837+0000 7f43af769640 1 -- 172.21.15.37:0/2289235496 >> 172.21.15.37:0/2289235496 conn(0x7f43a8071050 msgr2=0x7f43a8071460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:21.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.837+0000 7f43af769640 1 -- 172.21.15.37:0/2289235496 shutdown_connections 2024-09-17T04:33:21.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.837+0000 7f43af769640 1 -- 172.21.15.37:0/2289235496 wait complete. 2024-09-17T04:33:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.839+0000 7f43af769640 1 Processor -- start 2024-09-17T04:33:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.839+0000 7f43af769640 1 -- start start 2024-09-17T04:33:21.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.840+0000 7f43af769640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4394003d10 0x7f43a8151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.840+0000 7f43af769640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f43a8153380 0x7f43a80cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.840+0000 7f43af769640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43a80b3060 0x7f43a80b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.840+0000 7f43af769640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43a80d03c0 con 0x7f4394003d10 2024-09-17T04:33:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.840+0000 7f43af769640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f43a80d0240 con 0x7f43a80b3060 2024-09-17T04:33:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.840+0000 7f43af769640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f43a80d0540 con 0x7f43a8153380 2024-09-17T04:33:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.840+0000 7f43ad505640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4394003d10 0x7f43a8151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.840+0000 7f43ad505640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4394003d10 0x7f43a8151ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40670/0 (socket says 172.21.15.37:40670) 2024-09-17T04:33:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.840+0000 7f43ad505640 1 -- 172.21.15.37:0/432303204 learned_addr learned my addr 172.21.15.37:0/432303204 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.841+0000 7f43add06640 1 --2- 172.21.15.37:0/432303204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43a80b3060 0x7f43a80b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.841+0000 7f43acd04640 1 --2- 172.21.15.37:0/432303204 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f43a8153380 0x7f43a80cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.841+0000 7f43ad505640 1 -- 172.21.15.37:0/432303204 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f43a8153380 msgr2=0x7f43a80cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.841+0000 7f43ad505640 1 --2- 172.21.15.37:0/432303204 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f43a8153380 0x7f43a80cc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.841+0000 7f43ad505640 1 -- 172.21.15.37:0/432303204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43a80b3060 msgr2=0x7f43a80b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:21.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.841+0000 7f43ad505640 1 --2- 172.21.15.37:0/432303204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43a80b3060 0x7f43a80b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:21.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.841+0000 7f43ad505640 1 -- 172.21.15.37:0/432303204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a80b39b0 con 0x7f4394003d10 2024-09-17T04:33:21.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.841+0000 7f43ad505640 1 --2- 172.21.15.37:0/432303204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4394003d10 0x7f43a8151ec0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f43a4001830 tx=0x7f43a4006170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:21.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.841+0000 7f439dffb640 1 -- 172.21.15.37:0/432303204 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a4043070 con 0x7f4394003d10 2024-09-17T04:33:21.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.841+0000 7f439dffb640 1 -- 172.21.15.37:0/432303204 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43a4003af0 con 0x7f4394003d10 2024-09-17T04:33:21.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.841+0000 7f43af769640 1 -- 172.21.15.37:0/432303204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a81f34a0 con 0x7f4394003d10 2024-09-17T04:33:21.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.841+0000 7f439dffb640 1 -- 172.21.15.37:0/432303204 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a4033480 con 0x7f4394003d10 2024-09-17T04:33:21.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.841+0000 7f43af769640 1 -- 172.21.15.37:0/432303204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a81f7200 con 0x7f4394003d10 2024-09-17T04:33:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.842+0000 7f43af769640 1 -- 172.21.15.37:0/432303204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43a80727f0 con 0x7f4394003d10 2024-09-17T04:33:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.843+0000 7f439dffb640 1 -- 172.21.15.37:0/432303204 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f43a4046070 con 0x7f4394003d10 2024-09-17T04:33:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.844+0000 7f439dffb640 1 --2- 172.21.15.37:0/432303204 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f438c078bc0 0x7f438c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.844+0000 7f439dffb640 1 -- 172.21.15.37:0/432303204 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f43a40baba0 con 0x7f4394003d10 2024-09-17T04:33:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.844+0000 7f43acd04640 1 --2- 172.21.15.37:0/432303204 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f438c078bc0 0x7f438c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:21.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.845+0000 7f43acd04640 1 --2- 172.21.15.37:0/432303204 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f438c078bc0 0x7f438c07b0d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f43a8072bd0 tx=0x7f4398005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:21.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:21.848+0000 7f439dffb640 1 -- 172.21.15.37:0/432303204 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f43a80b39b0 con 0x7f4394003d10 2024-09-17T04:33:22.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.075+0000 7f43af769640 1 -- 172.21.15.37:0/432303204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f43a80b77f0 con 0x7f4394003d10 2024-09-17T04:33:22.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.699+0000 7f439dffb640 1 -- 172.21.15.37:0/432303204 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f43a4006260 con 0x7f4394003d10 2024-09-17T04:33:22.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.702+0000 7f43af769640 1 -- 172.21.15.37:0/432303204 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f438c078bc0 msgr2=0x7f438c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.702+0000 7f43af769640 1 --2- 172.21.15.37:0/432303204 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f438c078bc0 0x7f438c07b0d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f43a8072bd0 tx=0x7f4398005f90 comp rx=0 tx=0).stop 2024-09-17T04:33:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.702+0000 7f43af769640 1 -- 172.21.15.37:0/432303204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4394003d10 msgr2=0x7f43a8151ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.702+0000 7f43af769640 1 --2- 172.21.15.37:0/432303204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4394003d10 0x7f43a8151ec0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f43a4001830 tx=0x7f43a4006170 comp rx=0 tx=0).stop 2024-09-17T04:33:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.702+0000 7f43af769640 1 -- 172.21.15.37:0/432303204 shutdown_connections 2024-09-17T04:33:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.702+0000 7f43af769640 1 --2- 172.21.15.37:0/432303204 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f438c078bc0 0x7f438c07b0d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.702+0000 7f43af769640 1 --2- 172.21.15.37:0/432303204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43a80b3060 0x7f43a80b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.702+0000 7f43af769640 1 --2- 172.21.15.37:0/432303204 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f43a8153380 0x7f43a80cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.702+0000 7f43af769640 1 --2- 172.21.15.37:0/432303204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4394003d10 0x7f43a8151ec0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.702+0000 7f43af769640 1 -- 172.21.15.37:0/432303204 >> 172.21.15.37:0/432303204 conn(0x7f43a8071050 msgr2=0x7f43a80b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.702+0000 7f43af769640 1 -- 172.21.15.37:0/432303204 shutdown_connections 2024-09-17T04:33:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.702+0000 7f43af769640 1 -- 172.21.15.37:0/432303204 wait complete. 2024-09-17T04:33:22.716 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T04:33:22.716 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:33:22.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.841+0000 7fd457e69640 1 -- 172.21.15.37:0/2757415715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd450162550 msgr2=0x7fd450162960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:22.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.841+0000 7fd457e69640 1 --2- 172.21.15.37:0/2757415715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd450162550 0x7fd450162960 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd45006d530 tx=0x7fd44002a410 comp rx=0 tx=0).stop 2024-09-17T04:33:22.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.841+0000 7fd457e69640 1 -- 172.21.15.37:0/2757415715 shutdown_connections 2024-09-17T04:33:22.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.841+0000 7fd457e69640 1 --2- 172.21.15.37:0/2757415715 >> v2:172.21.15.92:3300/0 conn(0x7fd450162ea0 0x7fd4500ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:22.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.841+0000 7fd457e69640 1 --2- 172.21.15.37:0/2757415715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd450162550 0x7fd450162960 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:22.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.841+0000 7fd457e69640 1 -- 172.21.15.37:0/2757415715 >> 172.21.15.37:0/2757415715 conn(0x7fd45009d7a0 msgr2=0x7fd4500d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:22.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.841+0000 7fd457e69640 1 -- 172.21.15.37:0/2757415715 shutdown_connections 2024-09-17T04:33:22.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.841+0000 7fd457e69640 1 -- 172.21.15.37:0/2757415715 wait complete. 2024-09-17T04:33:22.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.843+0000 7fd457e69640 1 Processor -- start 2024-09-17T04:33:22.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.843+0000 7fd457e69640 1 -- start start 2024-09-17T04:33:22.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.844+0000 7fd457e69640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd4500d3ed0 0x7fd4501f9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:22.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.844+0000 7fd457e69640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd450162550 0x7fd4500b5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:22.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.844+0000 7fd457e69640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd450162ea0 0x7fd4500c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:22.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.844+0000 7fd457e69640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd4500cf1e0 con 0x7fd450162550 2024-09-17T04:33:22.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.844+0000 7fd457e69640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd4500cf060 con 0x7fd450162ea0 2024-09-17T04:33:22.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.844+0000 7fd457e69640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fd4500cf360 con 0x7fd4500d3ed0 2024-09-17T04:33:22.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.844+0000 7fd455404640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd450162550 0x7fd4500b5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:22.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.844+0000 7fd455404640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd450162550 0x7fd4500b5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40688/0 (socket says 172.21.15.37:40688) 2024-09-17T04:33:22.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.844+0000 7fd455404640 1 -- 172.21.15.37:0/1435673922 learned_addr learned my addr 172.21.15.37:0/1435673922 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:22.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.845+0000 7fd456406640 1 --2- 172.21.15.37:0/1435673922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd450162ea0 0x7fd4500c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:22.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.845+0000 7fd455c05640 1 --2- 172.21.15.37:0/1435673922 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd4500d3ed0 0x7fd4501f9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:22.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.845+0000 7fd455404640 1 -- 172.21.15.37:0/1435673922 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd4500d3ed0 msgr2=0x7fd4501f9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:22.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.845+0000 7fd455404640 1 --2- 172.21.15.37:0/1435673922 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd4500d3ed0 0x7fd4501f9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:22.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.845+0000 7fd455404640 1 -- 172.21.15.37:0/1435673922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd450162ea0 msgr2=0x7fd4500c9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:22.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.845+0000 7fd455404640 1 --2- 172.21.15.37:0/1435673922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd450162ea0 0x7fd4500c9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:22.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.845+0000 7fd455404640 1 -- 172.21.15.37:0/1435673922 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4501f6390 con 0x7fd450162550 2024-09-17T04:33:22.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.845+0000 7fd455404640 1 --2- 172.21.15.37:0/1435673922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd450162550 0x7fd4500b5230 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd44002aa00 tx=0x7fd440033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:22.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.845+0000 7fd4467fc640 1 -- 172.21.15.37:0/1435673922 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440005c90 con 0x7fd450162550 2024-09-17T04:33:22.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.845+0000 7fd4467fc640 1 -- 172.21.15.37:0/1435673922 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd440031e50 con 0x7fd450162550 2024-09-17T04:33:22.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.845+0000 7fd457e69640 1 -- 172.21.15.37:0/1435673922 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4501f55d0 con 0x7fd450162550 2024-09-17T04:33:22.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.845+0000 7fd4467fc640 1 -- 172.21.15.37:0/1435673922 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440005240 con 0x7fd450162550 2024-09-17T04:33:22.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.845+0000 7fd457e69640 1 -- 172.21.15.37:0/1435673922 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4501f5b90 con 0x7fd450162550 2024-09-17T04:33:22.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.846+0000 7fd457e69640 1 -- 172.21.15.37:0/1435673922 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4500b66a0 con 0x7fd450162550 2024-09-17T04:33:22.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.847+0000 7fd4467fc640 1 -- 172.21.15.37:0/1435673922 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd4400053e0 con 0x7fd450162550 2024-09-17T04:33:22.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.848+0000 7fd4467fc640 1 --2- 172.21.15.37:0/1435673922 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd434078bc0 0x7fd43407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:22.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.848+0000 7fd4467fc640 1 -- 172.21.15.37:0/1435673922 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd4400bbd40 con 0x7fd450162550 2024-09-17T04:33:22.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.848+0000 7fd455c05640 1 --2- 172.21.15.37:0/1435673922 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd434078bc0 0x7fd43407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:22.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.849+0000 7fd455c05640 1 --2- 172.21.15.37:0/1435673922 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd434078bc0 0x7fd43407b0d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd44c001760 tx=0x7fd44c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:22.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:22.852+0000 7fd4467fc640 1 -- 172.21.15.37:0/1435673922 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd44003f030 con 0x7fd450162550 2024-09-17T04:33:23.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.071+0000 7fd457e69640 1 -- 172.21.15.37:0/1435673922 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd450162960 con 0x7fd450162550 2024-09-17T04:33:23.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.072+0000 7fd4467fc640 1 -- 172.21.15.37:0/1435673922 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5217 (secure 0 0 0) 0x7fd440031770 con 0x7fd450162550 2024-09-17T04:33:23.072 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:23.072 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"btime":"2024-09-17T04:33:22:694087+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4387,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/4285801054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4285801054},{"type":"v1","addr":"172.21.15.92:6836","nonce":4285801054}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4401,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/3514084301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3514084301},{"type":"v1","addr":"172.21.15.37:6834","nonce":3514084301}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4404,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3229527106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":3229527106},{"type":"v1","addr":"172.21.15.37:6835","nonce":3229527106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:19.050573+0000","modified":"2024-09-17T04:33:22.076946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4419":{"gid":4419,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2128554253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2128554253},{"type":"v1","addr":"172.21.15.92:6837","nonce":2128554253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4419,"qdb_cluster":[4419]},"id":1}]} 2024-09-17T04:33:23.073 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-09-17T04:33:23.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.074+0000 7fd457e69640 1 -- 172.21.15.37:0/1435673922 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd434078bc0 msgr2=0x7fd43407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:23.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.074+0000 7fd457e69640 1 --2- 172.21.15.37:0/1435673922 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd434078bc0 0x7fd43407b0d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd44c001760 tx=0x7fd44c009040 comp rx=0 tx=0).stop 2024-09-17T04:33:23.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.074+0000 7fd457e69640 1 -- 172.21.15.37:0/1435673922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd450162550 msgr2=0x7fd4500b5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:23.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.074+0000 7fd457e69640 1 --2- 172.21.15.37:0/1435673922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd450162550 0x7fd4500b5230 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd44002aa00 tx=0x7fd440033c20 comp rx=0 tx=0).stop 2024-09-17T04:33:23.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.074+0000 7fd457e69640 1 -- 172.21.15.37:0/1435673922 shutdown_connections 2024-09-17T04:33:23.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.074+0000 7fd457e69640 1 --2- 172.21.15.37:0/1435673922 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd434078bc0 0x7fd43407b0d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.074+0000 7fd457e69640 1 --2- 172.21.15.37:0/1435673922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd450162ea0 0x7fd4500c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.074+0000 7fd457e69640 1 --2- 172.21.15.37:0/1435673922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd450162550 0x7fd4500b5230 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.074+0000 7fd457e69640 1 --2- 172.21.15.37:0/1435673922 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd4500d3ed0 0x7fd4501f9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.074+0000 7fd457e69640 1 -- 172.21.15.37:0/1435673922 >> 172.21.15.37:0/1435673922 conn(0x7fd45009d7a0 msgr2=0x7fd4500d0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:23.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.074+0000 7fd457e69640 1 -- 172.21.15.37:0/1435673922 shutdown_connections 2024-09-17T04:33:23.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.074+0000 7fd457e69640 1 -- 172.21.15.37:0/1435673922 wait complete. 2024-09-17T04:33:23.085 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:33:23.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.201+0000 7fa88abdf640 1 -- 172.21.15.37:0/3098858422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa88406d470 msgr2=0x7fa884071560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.201+0000 7fa88abdf640 1 --2- 172.21.15.37:0/3098858422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa88406d470 0x7fa884071560 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa88406a940 tx=0x7fa87c02aac0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.201+0000 7fa88abdf640 1 -- 172.21.15.37:0/3098858422 shutdown_connections 2024-09-17T04:33:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.201+0000 7fa88abdf640 1 --2- 172.21.15.37:0/3098858422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa88406d470 0x7fa884071560 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.201+0000 7fa88abdf640 1 --2- 172.21.15.37:0/3098858422 >> v2:172.21.15.92:3300/0 conn(0x7fa8840ce6d0 0x7fa8840cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.201+0000 7fa88abdf640 1 -- 172.21.15.37:0/3098858422 >> 172.21.15.37:0/3098858422 conn(0x7fa8840cdbb0 msgr2=0x7fa8840cdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.201+0000 7fa88abdf640 1 -- 172.21.15.37:0/3098858422 shutdown_connections 2024-09-17T04:33:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.201+0000 7fa88abdf640 1 -- 172.21.15.37:0/3098858422 wait complete. 2024-09-17T04:33:23.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.203+0000 7fa88abdf640 1 Processor -- start 2024-09-17T04:33:23.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.203+0000 7fa88abdf640 1 -- start start 2024-09-17T04:33:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.205+0000 7fa88abdf640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa8840ce6d0 0x7fa88414bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.205+0000 7fa88abdf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa88415eaf0 0x7fa8841f8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.205+0000 7fa88abdf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8840b4000 0x7fa8841f8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.205+0000 7fa88abdf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa884071aa0 con 0x7fa88415eaf0 2024-09-17T04:33:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.205+0000 7fa88abdf640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa88415f110 con 0x7fa8840b4000 2024-09-17T04:33:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.205+0000 7fa88abdf640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fa884071c20 con 0x7fa8840ce6d0 2024-09-17T04:33:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.205+0000 7fa883fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa88415eaf0 0x7fa8841f8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.205+0000 7fa883fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa88415eaf0 0x7fa8841f8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40718/0 (socket says 172.21.15.37:40718) 2024-09-17T04:33:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.205+0000 7fa883fff640 1 -- 172.21.15.37:0/124602190 learned_addr learned my addr 172.21.15.37:0/124602190 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.205+0000 7fa88897b640 1 --2- 172.21.15.37:0/124602190 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa8840ce6d0 0x7fa88414bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.205+0000 7fa88917c640 1 --2- 172.21.15.37:0/124602190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8840b4000 0x7fa8841f8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.205+0000 7fa883fff640 1 -- 172.21.15.37:0/124602190 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa8840ce6d0 msgr2=0x7fa88414bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.205+0000 7fa883fff640 1 --2- 172.21.15.37:0/124602190 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa8840ce6d0 0x7fa88414bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.205+0000 7fa883fff640 1 -- 172.21.15.37:0/124602190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8840b4000 msgr2=0x7fa8841f8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.205+0000 7fa883fff640 1 --2- 172.21.15.37:0/124602190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8840b4000 0x7fa8841f8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.205+0000 7fa883fff640 1 -- 172.21.15.37:0/124602190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8840b6e20 con 0x7fa88415eaf0 2024-09-17T04:33:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.205+0000 7fa883fff640 1 --2- 172.21.15.37:0/124602190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa88415eaf0 0x7fa8841f8a90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa87000c3d0 tx=0x7fa87000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.205+0000 7fa8817fa640 1 -- 172.21.15.37:0/124602190 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870017070 con 0x7fa88415eaf0 2024-09-17T04:33:23.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.205+0000 7fa8817fa640 1 -- 172.21.15.37:0/124602190 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8700053c0 con 0x7fa88415eaf0 2024-09-17T04:33:23.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.205+0000 7fa8817fa640 1 -- 172.21.15.37:0/124602190 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870003850 con 0x7fa88415eaf0 2024-09-17T04:33:23.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.205+0000 7fa88abdf640 1 -- 172.21.15.37:0/124602190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8840b6860 con 0x7fa88415eaf0 2024-09-17T04:33:23.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.206+0000 7fa88abdf640 1 -- 172.21.15.37:0/124602190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8841f6790 con 0x7fa88415eaf0 2024-09-17T04:33:23.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.206+0000 7fa88abdf640 1 -- 172.21.15.37:0/124602190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8840720d0 con 0x7fa88415eaf0 2024-09-17T04:33:23.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.207+0000 7fa8817fa640 1 -- 172.21.15.37:0/124602190 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa87001a070 con 0x7fa88415eaf0 2024-09-17T04:33:23.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.208+0000 7fa8817fa640 1 --2- 172.21.15.37:0/124602190 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa868078b90 0x7fa86807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:23.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.208+0000 7fa8817fa640 1 -- 172.21.15.37:0/124602190 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa870099fc0 con 0x7fa88415eaf0 2024-09-17T04:33:23.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.208+0000 7fa88897b640 1 --2- 172.21.15.37:0/124602190 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa868078b90 0x7fa86807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:23.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.208+0000 7fa88897b640 1 --2- 172.21.15.37:0/124602190 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa868078b90 0x7fa86807b0a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa878009b30 tx=0x7fa878005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:23.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.212+0000 7fa8817fa640 1 -- 172.21.15.37:0/124602190 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa870061e40 con 0x7fa88415eaf0 2024-09-17T04:33:23.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.400+0000 7fa88abdf640 1 -- 172.21.15.37:0/124602190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8840b4500 con 0x7fa88415eaf0 2024-09-17T04:33:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.401+0000 7fa8817fa640 1 -- 172.21.15.37:0/124602190 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16605 (secure 0 0 0) 0x7fa870066870 con 0x7fa88415eaf0 2024-09-17T04:33:23.401 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:23.401 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":22,"fsid":"a908a603-430c-4209-8294-0c3460b35f42","created":"2024-09-17T04:32:54.139221+0000","modified":"2024-09-17T04:33:20.677493+0000","last_up_change":"2024-09-17T04:33:06.922210+0000","last_in_change":"2024-09-17T04:32:57.754118+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:33:08.461907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:33:16.958785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:18.013768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ea4b8d1-0d22-4dd9-b913-9df585ef24ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6809","nonce":2283587593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6811","nonce":2283587593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6815","nonce":2283587593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6813","nonce":2283587593}]},"public_addr":"172.21.15.37:6809/2283587593","cluster_addr":"172.21.15.37:6811/2283587593","heartbeat_back_addr":"172.21.15.37:6815/2283587593","heartbeat_front_addr":"172.21.15.37:6813/2283587593","state":["exists","up"]},{"osd":1,"uuid":"ac745a33-e8ad-4083-9c18-71bf81982b76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6825","nonce":2657955935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6827","nonce":2657955935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6831","nonce":2657955935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6829","nonce":2657955935}]},"public_addr":"172.21.15.37:6825/2657955935","cluster_addr":"172.21.15.37:6827/2657955935","heartbeat_back_addr":"172.21.15.37:6831/2657955935","heartbeat_front_addr":"172.21.15.37:6829/2657955935","state":["exists","up"]},{"osd":2,"uuid":"3c63f3d8-132a-4735-b0bc-d64313a46503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6817","nonce":579720176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6819","nonce":579720176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6823","nonce":579720176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6821","nonce":579720176}]},"public_addr":"172.21.15.37:6817/579720176","cluster_addr":"172.21.15.37:6819/579720176","heartbeat_back_addr":"172.21.15.37:6823/579720176","heartbeat_front_addr":"172.21.15.37:6821/579720176","state":["exists","up"]},{"osd":3,"uuid":"4f8c6ca2-ea1d-4610-9127-52c4dd8bc8e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6801","nonce":809975826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6803","nonce":809975826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6807","nonce":809975826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6805","nonce":809975826}]},"public_addr":"172.21.15.37:6801/809975826","cluster_addr":"172.21.15.37:6803/809975826","heartbeat_back_addr":"172.21.15.37:6807/809975826","heartbeat_front_addr":"172.21.15.37:6805/809975826","state":["exists","up"]},{"osd":4,"uuid":"63d0c4ab-f115-47b4-9611-12a10153473a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6819","nonce":2509785752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6821","nonce":2509785752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6825","nonce":2509785752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6823","nonce":2509785752}]},"public_addr":"172.21.15.92:6819/2509785752","cluster_addr":"172.21.15.92:6821/2509785752","heartbeat_back_addr":"172.21.15.92:6825/2509785752","heartbeat_front_addr":"172.21.15.92:6823/2509785752","state":["exists","up"]},{"osd":5,"uuid":"4c56c04d-0c07-4f76-9d75-6335121310ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6827","nonce":248263832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6829","nonce":248263832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6833","nonce":248263832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6831","nonce":248263832}]},"public_addr":"172.21.15.92:6827/248263832","cluster_addr":"172.21.15.92:6829/248263832","heartbeat_back_addr":"172.21.15.92:6833/248263832","heartbeat_front_addr":"172.21.15.92:6831/248263832","state":["exists","up"]},{"osd":6,"uuid":"b30eae7c-096a-4d24-ac51-96b8b13ecaee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6811","nonce":4078365985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6813","nonce":4078365985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6817","nonce":4078365985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6815","nonce":4078365985}]},"public_addr":"172.21.15.92:6811/4078365985","cluster_addr":"172.21.15.92:6813/4078365985","heartbeat_back_addr":"172.21.15.92:6817/4078365985","heartbeat_front_addr":"172.21.15.92:6815/4078365985","state":["exists","up"]},{"osd":7,"uuid":"62f12540-d374-4a4c-9ff7-63b7304fe77f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6803","nonce":3444787223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6805","nonce":3444787223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6809","nonce":3444787223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6807","nonce":3444787223}]},"public_addr":"172.21.15.92:6803/3444787223","cluster_addr":"172.21.15.92:6805/3444787223","heartbeat_back_addr":"172.21.15.92:6809/3444787223","heartbeat_front_addr":"172.21.15.92:6807/3444787223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.006420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.111135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.103325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.072656+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.200989+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.060455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.774773+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.865778+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:33:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.404+0000 7fa88abdf640 1 -- 172.21.15.37:0/124602190 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa868078b90 msgr2=0x7fa86807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.404+0000 7fa88abdf640 1 --2- 172.21.15.37:0/124602190 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa868078b90 0x7fa86807b0a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa878009b30 tx=0x7fa878005f90 comp rx=0 tx=0).stop 2024-09-17T04:33:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.404+0000 7fa88abdf640 1 -- 172.21.15.37:0/124602190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa88415eaf0 msgr2=0x7fa8841f8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.404+0000 7fa88abdf640 1 --2- 172.21.15.37:0/124602190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa88415eaf0 0x7fa8841f8a90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa87000c3d0 tx=0x7fa87000c890 comp rx=0 tx=0).stop 2024-09-17T04:33:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.404+0000 7fa88abdf640 1 -- 172.21.15.37:0/124602190 shutdown_connections 2024-09-17T04:33:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.404+0000 7fa88abdf640 1 --2- 172.21.15.37:0/124602190 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa868078b90 0x7fa86807b0a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.404+0000 7fa88abdf640 1 --2- 172.21.15.37:0/124602190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8840b4000 0x7fa8841f8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.404+0000 7fa88abdf640 1 --2- 172.21.15.37:0/124602190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa88415eaf0 0x7fa8841f8a90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.404+0000 7fa88abdf640 1 --2- 172.21.15.37:0/124602190 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa8840ce6d0 0x7fa88414bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.404+0000 7fa88abdf640 1 -- 172.21.15.37:0/124602190 >> 172.21.15.37:0/124602190 conn(0x7fa8840cdbb0 msgr2=0x7fa8840cc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.404+0000 7fa88abdf640 1 -- 172.21.15.37:0/124602190 shutdown_connections 2024-09-17T04:33:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.404+0000 7fa88abdf640 1 -- 172.21.15.37:0/124602190 wait complete. 2024-09-17T04:33:23.415 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T04:33:23.415 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:33:23.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.532+0000 7fb79c9a8640 1 -- 172.21.15.37:0/217413165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb784004800 msgr2=0x7fb784004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:23.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.532+0000 7fb79c9a8640 1 --2- 172.21.15.37:0/217413165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb784004800 0x7fb784004cd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb780006f40 tx=0x7fb784005830 comp rx=0 tx=0).stop 2024-09-17T04:33:23.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.532+0000 7fb796d9d640 1 -- 172.21.15.37:0/217413165 reap_dead start 2024-09-17T04:33:23.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.532+0000 7fb79c9a8640 1 -- 172.21.15.37:0/217413165 shutdown_connections 2024-09-17T04:33:23.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.532+0000 7fb79c9a8640 1 -- 172.21.15.37:0/217413165 >> 172.21.15.37:0/217413165 conn(0x7fb7980d4540 msgr2=0x7fb7980d4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:23.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.532+0000 7fb79c9a8640 1 -- 172.21.15.37:0/217413165 shutdown_connections 2024-09-17T04:33:23.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.532+0000 7fb79c9a8640 1 -- 172.21.15.37:0/217413165 wait complete. 2024-09-17T04:33:23.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.534+0000 7fb79c9a8640 1 Processor -- start 2024-09-17T04:33:23.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.534+0000 7fb79c9a8640 1 -- start start 2024-09-17T04:33:23.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.535+0000 7fb79c9a8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb784004800 0x7fb79815b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:23.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.536+0000 7fb79c9a8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7980cdcf0 0x7fb798159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:23.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.536+0000 7fb79c9a8640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb7980ce100 0x7fb798159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:23.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.536+0000 7fb79c9a8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb7981631c0 con 0x7fb7980cdcf0 2024-09-17T04:33:23.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.536+0000 7fb79c9a8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb7981675f0 con 0x7fb784004800 2024-09-17T04:33:23.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.536+0000 7fb79c9a8640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fb798142000 con 0x7fb7980ce100 2024-09-17T04:33:23.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.536+0000 7fb78dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7980cdcf0 0x7fb798159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:23.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.536+0000 7fb78dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7980cdcf0 0x7fb798159820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40730/0 (socket says 172.21.15.37:40730) 2024-09-17T04:33:23.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.536+0000 7fb78dd9b640 1 -- 172.21.15.37:0/1693975955 learned_addr learned my addr 172.21.15.37:0/1693975955 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:23.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.536+0000 7fb79659c640 1 --2- 172.21.15.37:0/1693975955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb784004800 0x7fb79815b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:23.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.536+0000 7fb796d9d640 1 --2- 172.21.15.37:0/1693975955 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb7980ce100 0x7fb798159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:23.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.536+0000 7fb78dd9b640 1 -- 172.21.15.37:0/1693975955 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb7980ce100 msgr2=0x7fb798159d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:23.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.536+0000 7fb78dd9b640 1 --2- 172.21.15.37:0/1693975955 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb7980ce100 0x7fb798159d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.536+0000 7fb78dd9b640 1 -- 172.21.15.37:0/1693975955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb784004800 msgr2=0x7fb79815b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:23.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.536+0000 7fb78dd9b640 1 --2- 172.21.15.37:0/1693975955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb784004800 0x7fb79815b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.536+0000 7fb78dd9b640 1 -- 172.21.15.37:0/1693975955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7981448c0 con 0x7fb7980cdcf0 2024-09-17T04:33:23.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.536+0000 7fb78dd9b640 1 --2- 172.21.15.37:0/1693975955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7980cdcf0 0x7fb798159820 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb780031040 tx=0x7fb780001cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:23.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.536+0000 7fb78f7fe640 1 -- 172.21.15.37:0/1693975955 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780003800 con 0x7fb7980cdcf0 2024-09-17T04:33:23.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.536+0000 7fb78f7fe640 1 -- 172.21.15.37:0/1693975955 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb780003a90 con 0x7fb7980cdcf0 2024-09-17T04:33:23.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.536+0000 7fb79c9a8640 1 -- 172.21.15.37:0/1693975955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7980b41d0 con 0x7fb7980cdcf0 2024-09-17T04:33:23.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.536+0000 7fb78f7fe640 1 -- 172.21.15.37:0/1693975955 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78002e440 con 0x7fb7980cdcf0 2024-09-17T04:33:23.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.536+0000 7fb79c9a8640 1 -- 172.21.15.37:0/1693975955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7980b4790 con 0x7fb7980cdcf0 2024-09-17T04:33:23.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.538+0000 7fb79c9a8640 1 -- 172.21.15.37:0/1693975955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb798069d10 con 0x7fb7980cdcf0 2024-09-17T04:33:23.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.538+0000 7fb78f7fe640 1 -- 172.21.15.37:0/1693975955 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb780035020 con 0x7fb7980cdcf0 2024-09-17T04:33:23.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.539+0000 7fb78f7fe640 1 --2- 172.21.15.37:0/1693975955 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb778078b90 0x7fb77807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:23.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.539+0000 7fb78f7fe640 1 -- 172.21.15.37:0/1693975955 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb7800baff0 con 0x7fb7980cdcf0 2024-09-17T04:33:23.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.539+0000 7fb79659c640 1 --2- 172.21.15.37:0/1693975955 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb778078b90 0x7fb77807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:23.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.540+0000 7fb79659c640 1 --2- 172.21.15.37:0/1693975955 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb778078b90 0x7fb77807b0a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb788006560 tx=0x7fb788006b10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:23.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.544+0000 7fb78f7fe640 1 -- 172.21.15.37:0/1693975955 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb798069d10 con 0x7fb7980cdcf0 2024-09-17T04:33:23.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.725+0000 7fb79c9a8640 1 -- 172.21.15.37:0/1693975955 --> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb79806a1a0 con 0x7fb778078b90 2024-09-17T04:33:23.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.730+0000 7fb78f7fe640 1 -- 172.21.15.37:0/1693975955 <== mgr.4100 v2:172.21.15.92:6800/864337835 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251934 (secure 0 0 0) 0x7fb7880014f0 con 0x7fb778078b90 2024-09-17T04:33:23.734 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:23.735 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-17T04:33:23.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.734+0000 7fb79c9a8640 1 -- 172.21.15.37:0/1693975955 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb778078b90 msgr2=0x7fb77807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:23.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.734+0000 7fb79c9a8640 1 --2- 172.21.15.37:0/1693975955 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb778078b90 0x7fb77807b0a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb788006560 tx=0x7fb788006b10 comp rx=0 tx=0).stop 2024-09-17T04:33:23.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.734+0000 7fb79c9a8640 1 -- 172.21.15.37:0/1693975955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7980cdcf0 msgr2=0x7fb798159820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:23.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.734+0000 7fb79c9a8640 1 --2- 172.21.15.37:0/1693975955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7980cdcf0 0x7fb798159820 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb780031040 tx=0x7fb780001cd0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.734+0000 7fb79c9a8640 1 -- 172.21.15.37:0/1693975955 shutdown_connections 2024-09-17T04:33:23.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.734+0000 7fb79c9a8640 1 --2- 172.21.15.37:0/1693975955 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb778078b90 0x7fb77807b0a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.734+0000 7fb79c9a8640 1 --2- 172.21.15.37:0/1693975955 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb7980ce100 0x7fb798159d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.734+0000 7fb79c9a8640 1 --2- 172.21.15.37:0/1693975955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7980cdcf0 0x7fb798159820 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.734+0000 7fb79c9a8640 1 --2- 172.21.15.37:0/1693975955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb784004800 0x7fb79815b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.734+0000 7fb79c9a8640 1 -- 172.21.15.37:0/1693975955 >> 172.21.15.37:0/1693975955 conn(0x7fb7980d4540 msgr2=0x7fb7980b6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:23.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.734+0000 7fb79c9a8640 1 -- 172.21.15.37:0/1693975955 shutdown_connections 2024-09-17T04:33:23.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.734+0000 7fb79c9a8640 1 -- 172.21.15.37:0/1693975955 wait complete. 2024-09-17T04:33:23.745 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-17T04:33:22.445928+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":129,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219212,"kb_used_data":2972,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755508,"statfs":{"total":773094113280,"available":772869640192,"internally_reserved":0,"allocated":3043328,"data_stored":1810520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.786082"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691577+0000","last_change":"2024-09-17T04:33:19.693609+0000","last_active":"2024-09-17T04:33:20.691577+0000","last_peered":"2024-09-17T04:33:20.691577+0000","last_clean":"2024-09-17T04:33:20.691577+0000","last_became_active":"2024-09-17T04:33:19.693159+0000","last_became_peered":"2024-09-17T04:33:19.693159+0000","last_unstale":"2024-09-17T04:33:20.691577+0000","last_undegraded":"2024-09-17T04:33:20.691577+0000","last_fullsized":"2024-09-17T04:33:20.691577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:29.058321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674309+0000","last_change":"2024-09-17T04:33:19.686724+0000","last_active":"2024-09-17T04:33:21.674309+0000","last_peered":"2024-09-17T04:33:21.674309+0000","last_clean":"2024-09-17T04:33:21.674309+0000","last_became_active":"2024-09-17T04:33:19.685840+0000","last_became_peered":"2024-09-17T04:33:19.685840+0000","last_unstale":"2024-09-17T04:33:21.674309+0000","last_undegraded":"2024-09-17T04:33:21.674309+0000","last_fullsized":"2024-09-17T04:33:21.674309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:21.722762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035198+0000","last_change":"2024-09-17T04:33:19.688324+0000","last_active":"2024-09-17T04:33:22.035198+0000","last_peered":"2024-09-17T04:33:22.035198+0000","last_clean":"2024-09-17T04:33:22.035198+0000","last_became_active":"2024-09-17T04:33:19.687914+0000","last_became_peered":"2024-09-17T04:33:19.687914+0000","last_unstale":"2024-09-17T04:33:22.035198+0000","last_undegraded":"2024-09-17T04:33:22.035198+0000","last_fullsized":"2024-09-17T04:33:22.035198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:44.753208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930207+0000","last_change":"2024-09-17T04:33:19.694987+0000","last_active":"2024-09-17T04:33:20.930207+0000","last_peered":"2024-09-17T04:33:20.930207+0000","last_clean":"2024-09-17T04:33:20.930207+0000","last_became_active":"2024-09-17T04:33:19.694651+0000","last_became_peered":"2024-09-17T04:33:19.694651+0000","last_unstale":"2024-09-17T04:33:20.930207+0000","last_undegraded":"2024-09-17T04:33:20.930207+0000","last_fullsized":"2024-09-17T04:33:20.930207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:57:44.135139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690540+0000","last_change":"2024-09-17T04:33:19.693592+0000","last_active":"2024-09-17T04:33:20.690540+0000","last_peered":"2024-09-17T04:33:20.690540+0000","last_clean":"2024-09-17T04:33:20.690540+0000","last_became_active":"2024-09-17T04:33:19.690470+0000","last_became_peered":"2024-09-17T04:33:19.690470+0000","last_unstale":"2024-09-17T04:33:20.690540+0000","last_undegraded":"2024-09-17T04:33:20.690540+0000","last_fullsized":"2024-09-17T04:33:20.690540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:16:38.450837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.929991+0000","last_change":"2024-09-17T04:33:19.696808+0000","last_active":"2024-09-17T04:33:20.929991+0000","last_peered":"2024-09-17T04:33:20.929991+0000","last_clean":"2024-09-17T04:33:20.929991+0000","last_became_active":"2024-09-17T04:33:19.696523+0000","last_became_peered":"2024-09-17T04:33:19.696523+0000","last_unstale":"2024-09-17T04:33:20.929991+0000","last_undegraded":"2024-09-17T04:33:20.929991+0000","last_fullsized":"2024-09-17T04:33:20.929991+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:21.722762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.689753+0000","last_change":"2024-09-17T04:33:19.698772+0000","last_active":"2024-09-17T04:33:20.689753+0000","last_peered":"2024-09-17T04:33:20.689753+0000","last_clean":"2024-09-17T04:33:20.689753+0000","last_became_active":"2024-09-17T04:33:19.698190+0000","last_became_peered":"2024-09-17T04:33:19.698190+0000","last_unstale":"2024-09-17T04:33:20.689753+0000","last_undegraded":"2024-09-17T04:33:20.689753+0000","last_fullsized":"2024-09-17T04:33:20.689753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:20.825186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035054+0000","last_change":"2024-09-17T04:33:19.702702+0000","last_active":"2024-09-17T04:33:22.035054+0000","last_peered":"2024-09-17T04:33:22.035054+0000","last_clean":"2024-09-17T04:33:22.035054+0000","last_became_active":"2024-09-17T04:33:19.701450+0000","last_became_peered":"2024-09-17T04:33:19.701450+0000","last_unstale":"2024-09-17T04:33:22.035054+0000","last_undegraded":"2024-09-17T04:33:22.035054+0000","last_fullsized":"2024-09-17T04:33:22.035054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:27.889015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691647+0000","last_change":"2024-09-17T04:33:19.702493+0000","last_active":"2024-09-17T04:33:20.691647+0000","last_peered":"2024-09-17T04:33:20.691647+0000","last_clean":"2024-09-17T04:33:20.691647+0000","last_became_active":"2024-09-17T04:33:19.702105+0000","last_became_peered":"2024-09-17T04:33:19.702105+0000","last_unstale":"2024-09-17T04:33:20.691647+0000","last_undegraded":"2024-09-17T04:33:20.691647+0000","last_fullsized":"2024-09-17T04:33:20.691647+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:21.242559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677475+0000","last_change":"2024-09-17T04:33:19.691454+0000","last_active":"2024-09-17T04:33:21.677475+0000","last_peered":"2024-09-17T04:33:21.677475+0000","last_clean":"2024-09-17T04:33:21.677475+0000","last_became_active":"2024-09-17T04:33:19.689654+0000","last_became_peered":"2024-09-17T04:33:19.689654+0000","last_unstale":"2024-09-17T04:33:21.677475+0000","last_undegraded":"2024-09-17T04:33:21.677475+0000","last_fullsized":"2024-09-17T04:33:21.677475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:49.699088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677244+0000","last_change":"2024-09-17T04:33:19.687918+0000","last_active":"2024-09-17T04:33:21.677244+0000","last_peered":"2024-09-17T04:33:21.677244+0000","last_clean":"2024-09-17T04:33:21.677244+0000","last_became_active":"2024-09-17T04:33:19.687575+0000","last_became_peered":"2024-09-17T04:33:19.687575+0000","last_unstale":"2024-09-17T04:33:21.677244+0000","last_undegraded":"2024-09-17T04:33:21.677244+0000","last_fullsized":"2024-09-17T04:33:21.677244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:39.340136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691463+0000","last_change":"2024-09-17T04:33:19.687282+0000","last_active":"2024-09-17T04:33:20.691463+0000","last_peered":"2024-09-17T04:33:20.691463+0000","last_clean":"2024-09-17T04:33:20.691463+0000","last_became_active":"2024-09-17T04:33:19.686308+0000","last_became_peered":"2024-09-17T04:33:19.686308+0000","last_unstale":"2024-09-17T04:33:20.691463+0000","last_undegraded":"2024-09-17T04:33:20.691463+0000","last_fullsized":"2024-09-17T04:33:20.691463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:05.880115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036019+0000","last_change":"2024-09-17T04:33:19.685671+0000","last_active":"2024-09-17T04:33:22.036019+0000","last_peered":"2024-09-17T04:33:22.036019+0000","last_clean":"2024-09-17T04:33:22.036019+0000","last_became_active":"2024-09-17T04:33:19.685339+0000","last_became_peered":"2024-09-17T04:33:19.685339+0000","last_unstale":"2024-09-17T04:33:22.036019+0000","last_undegraded":"2024-09-17T04:33:22.036019+0000","last_fullsized":"2024-09-17T04:33:22.036019+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:20.825186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036606+0000","last_change":"2024-09-17T04:33:19.682161+0000","last_active":"2024-09-17T04:33:22.036606+0000","last_peered":"2024-09-17T04:33:22.036606+0000","last_clean":"2024-09-17T04:33:22.036606+0000","last_became_active":"2024-09-17T04:33:19.679848+0000","last_became_peered":"2024-09-17T04:33:19.679848+0000","last_unstale":"2024-09-17T04:33:22.036606+0000","last_undegraded":"2024-09-17T04:33:22.036606+0000","last_fullsized":"2024-09-17T04:33:22.036606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:49.699088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674099+0000","last_change":"2024-09-17T04:33:19.681771+0000","last_active":"2024-09-17T04:33:21.674099+0000","last_peered":"2024-09-17T04:33:21.674099+0000","last_clean":"2024-09-17T04:33:21.674099+0000","last_became_active":"2024-09-17T04:33:19.681448+0000","last_became_peered":"2024-09-17T04:33:19.681448+0000","last_unstale":"2024-09-17T04:33:21.674099+0000","last_undegraded":"2024-09-17T04:33:21.674099+0000","last_fullsized":"2024-09-17T04:33:21.674099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:19.224546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.929925+0000","last_change":"2024-09-17T04:33:19.701067+0000","last_active":"2024-09-17T04:33:20.929925+0000","last_peered":"2024-09-17T04:33:20.929925+0000","last_clean":"2024-09-17T04:33:20.929925+0000","last_became_active":"2024-09-17T04:33:19.698312+0000","last_became_peered":"2024-09-17T04:33:19.698312+0000","last_unstale":"2024-09-17T04:33:20.929925+0000","last_undegraded":"2024-09-17T04:33:20.929925+0000","last_fullsized":"2024-09-17T04:33:20.929925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:49.699088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930842+0000","last_change":"2024-09-17T04:33:19.698864+0000","last_active":"2024-09-17T04:33:20.930842+0000","last_peered":"2024-09-17T04:33:20.930842+0000","last_clean":"2024-09-17T04:33:20.930842+0000","last_became_active":"2024-09-17T04:33:19.698394+0000","last_became_peered":"2024-09-17T04:33:19.698394+0000","last_unstale":"2024-09-17T04:33:20.930842+0000","last_undegraded":"2024-09-17T04:33:20.930842+0000","last_fullsized":"2024-09-17T04:33:20.930842+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:16:06.475955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.684437+0000","last_change":"2024-09-17T04:33:19.692126+0000","last_active":"2024-09-17T04:33:20.684437+0000","last_peered":"2024-09-17T04:33:20.684437+0000","last_clean":"2024-09-17T04:33:20.684437+0000","last_became_active":"2024-09-17T04:33:19.691759+0000","last_became_peered":"2024-09-17T04:33:19.691759+0000","last_unstale":"2024-09-17T04:33:20.684437+0000","last_undegraded":"2024-09-17T04:33:20.684437+0000","last_fullsized":"2024-09-17T04:33:20.684437+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:48.205935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690446+0000","last_change":"2024-09-17T04:33:19.693246+0000","last_active":"2024-09-17T04:33:20.690446+0000","last_peered":"2024-09-17T04:33:20.690446+0000","last_clean":"2024-09-17T04:33:20.690446+0000","last_became_active":"2024-09-17T04:33:19.692591+0000","last_became_peered":"2024-09-17T04:33:19.692591+0000","last_unstale":"2024-09-17T04:33:20.690446+0000","last_undegraded":"2024-09-17T04:33:20.690446+0000","last_fullsized":"2024-09-17T04:33:20.690446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:58.709184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677180+0000","last_change":"2024-09-17T04:33:19.684484+0000","last_active":"2024-09-17T04:33:21.677180+0000","last_peered":"2024-09-17T04:33:21.677180+0000","last_clean":"2024-09-17T04:33:21.677180+0000","last_became_active":"2024-09-17T04:33:19.683795+0000","last_became_peered":"2024-09-17T04:33:19.683795+0000","last_unstale":"2024-09-17T04:33:21.677180+0000","last_undegraded":"2024-09-17T04:33:21.677180+0000","last_fullsized":"2024-09-17T04:33:21.677180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:03.560173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036302+0000","last_change":"2024-09-17T04:33:19.691536+0000","last_active":"2024-09-17T04:33:22.036302+0000","last_peered":"2024-09-17T04:33:22.036302+0000","last_clean":"2024-09-17T04:33:22.036302+0000","last_became_active":"2024-09-17T04:33:19.691142+0000","last_became_peered":"2024-09-17T04:33:19.691142+0000","last_unstale":"2024-09-17T04:33:22.036302+0000","last_undegraded":"2024-09-17T04:33:22.036302+0000","last_fullsized":"2024-09-17T04:33:22.036302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:12.894886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930139+0000","last_change":"2024-09-17T04:33:19.700787+0000","last_active":"2024-09-17T04:33:20.930139+0000","last_peered":"2024-09-17T04:33:20.930139+0000","last_clean":"2024-09-17T04:33:20.930139+0000","last_became_active":"2024-09-17T04:33:19.700430+0000","last_became_peered":"2024-09-17T04:33:19.700430+0000","last_unstale":"2024-09-17T04:33:20.930139+0000","last_undegraded":"2024-09-17T04:33:20.930139+0000","last_fullsized":"2024-09-17T04:33:20.930139+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:14.652736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035743+0000","last_change":"2024-09-17T04:33:19.684011+0000","last_active":"2024-09-17T04:33:22.035743+0000","last_peered":"2024-09-17T04:33:22.035743+0000","last_clean":"2024-09-17T04:33:22.035743+0000","last_became_active":"2024-09-17T04:33:19.683695+0000","last_became_peered":"2024-09-17T04:33:19.683695+0000","last_unstale":"2024-09-17T04:33:22.035743+0000","last_undegraded":"2024-09-17T04:33:22.035743+0000","last_fullsized":"2024-09-17T04:33:22.035743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:11.769729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930009+0000","last_change":"2024-09-17T04:33:19.689591+0000","last_active":"2024-09-17T04:33:20.930009+0000","last_peered":"2024-09-17T04:33:20.930009+0000","last_clean":"2024-09-17T04:33:20.930009+0000","last_became_active":"2024-09-17T04:33:19.689263+0000","last_became_peered":"2024-09-17T04:33:19.689263+0000","last_unstale":"2024-09-17T04:33:20.930009+0000","last_undegraded":"2024-09-17T04:33:20.930009+0000","last_fullsized":"2024-09-17T04:33:20.930009+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:06:04.595656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035649+0000","last_change":"2024-09-17T04:33:19.689915+0000","last_active":"2024-09-17T04:33:22.035649+0000","last_peered":"2024-09-17T04:33:22.035649+0000","last_clean":"2024-09-17T04:33:22.035649+0000","last_became_active":"2024-09-17T04:33:19.689612+0000","last_became_peered":"2024-09-17T04:33:19.689612+0000","last_unstale":"2024-09-17T04:33:22.035649+0000","last_undegraded":"2024-09-17T04:33:22.035649+0000","last_fullsized":"2024-09-17T04:33:22.035649+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:39.340136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677482+0000","last_change":"2024-09-17T04:33:19.695306+0000","last_active":"2024-09-17T04:33:21.677482+0000","last_peered":"2024-09-17T04:33:21.677482+0000","last_clean":"2024-09-17T04:33:21.677482+0000","last_became_active":"2024-09-17T04:33:19.694980+0000","last_became_peered":"2024-09-17T04:33:19.694980+0000","last_unstale":"2024-09-17T04:33:21.677482+0000","last_undegraded":"2024-09-17T04:33:21.677482+0000","last_fullsized":"2024-09-17T04:33:21.677482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:11.769729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674250+0000","last_change":"2024-09-17T04:33:19.684015+0000","last_active":"2024-09-17T04:33:21.674250+0000","last_peered":"2024-09-17T04:33:21.674250+0000","last_clean":"2024-09-17T04:33:21.674250+0000","last_became_active":"2024-09-17T04:33:19.683668+0000","last_became_peered":"2024-09-17T04:33:19.683668+0000","last_unstale":"2024-09-17T04:33:21.674250+0000","last_undegraded":"2024-09-17T04:33:21.674250+0000","last_fullsized":"2024-09-17T04:33:21.674250+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:49.699088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035804+0000","last_change":"2024-09-17T04:33:19.692206+0000","last_active":"2024-09-17T04:33:22.035804+0000","last_peered":"2024-09-17T04:33:22.035804+0000","last_clean":"2024-09-17T04:33:22.035804+0000","last_became_active":"2024-09-17T04:33:19.691739+0000","last_became_peered":"2024-09-17T04:33:19.691739+0000","last_unstale":"2024-09-17T04:33:22.035804+0000","last_undegraded":"2024-09-17T04:33:22.035804+0000","last_fullsized":"2024-09-17T04:33:22.035804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:44.305489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036443+0000","last_change":"2024-09-17T04:33:19.688554+0000","last_active":"2024-09-17T04:33:22.036443+0000","last_peered":"2024-09-17T04:33:22.036443+0000","last_clean":"2024-09-17T04:33:22.036443+0000","last_became_active":"2024-09-17T04:33:19.688216+0000","last_became_peered":"2024-09-17T04:33:19.688216+0000","last_unstale":"2024-09-17T04:33:22.036443+0000","last_undegraded":"2024-09-17T04:33:22.036443+0000","last_fullsized":"2024-09-17T04:33:22.036443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:19.224546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930414+0000","last_change":"2024-09-17T04:33:19.690117+0000","last_active":"2024-09-17T04:33:20.930414+0000","last_peered":"2024-09-17T04:33:20.930414+0000","last_clean":"2024-09-17T04:33:20.930414+0000","last_became_active":"2024-09-17T04:33:19.689494+0000","last_became_peered":"2024-09-17T04:33:19.689494+0000","last_unstale":"2024-09-17T04:33:20.930414+0000","last_undegraded":"2024-09-17T04:33:20.930414+0000","last_fullsized":"2024-09-17T04:33:20.930414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:12.356559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.684135+0000","last_change":"2024-09-17T04:33:19.702680+0000","last_active":"2024-09-17T04:33:20.684135+0000","last_peered":"2024-09-17T04:33:20.684135+0000","last_clean":"2024-09-17T04:33:20.684135+0000","last_became_active":"2024-09-17T04:33:19.702240+0000","last_became_peered":"2024-09-17T04:33:19.702240+0000","last_unstale":"2024-09-17T04:33:20.684135+0000","last_undegraded":"2024-09-17T04:33:20.684135+0000","last_fullsized":"2024-09-17T04:33:20.684135+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:56:27.523263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677072+0000","last_change":"2024-09-17T04:33:19.698110+0000","last_active":"2024-09-17T04:33:21.677072+0000","last_peered":"2024-09-17T04:33:21.677072+0000","last_clean":"2024-09-17T04:33:21.677072+0000","last_became_active":"2024-09-17T04:33:19.697800+0000","last_became_peered":"2024-09-17T04:33:19.697800+0000","last_unstale":"2024-09-17T04:33:21.677072+0000","last_undegraded":"2024-09-17T04:33:21.677072+0000","last_fullsized":"2024-09-17T04:33:21.677072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:27.889015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.689882+0000","last_change":"2024-09-17T04:33:19.682605+0000","last_active":"2024-09-17T04:33:20.689882+0000","last_peered":"2024-09-17T04:33:20.689882+0000","last_clean":"2024-09-17T04:33:20.689882+0000","last_became_active":"2024-09-17T04:33:19.682212+0000","last_became_peered":"2024-09-17T04:33:19.682212+0000","last_unstale":"2024-09-17T04:33:20.689882+0000","last_undegraded":"2024-09-17T04:33:20.689882+0000","last_fullsized":"2024-09-17T04:33:20.689882+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:36:00.513397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036154+0000","last_change":"2024-09-17T04:33:18.690638+0000","last_active":"2024-09-17T04:33:22.036154+0000","last_peered":"2024-09-17T04:33:22.036154+0000","last_clean":"2024-09-17T04:33:22.036154+0000","last_became_active":"2024-09-17T04:33:18.690326+0000","last_became_peered":"2024-09-17T04:33:18.690326+0000","last_unstale":"2024-09-17T04:33:22.036154+0000","last_undegraded":"2024-09-17T04:33:22.036154+0000","last_fullsized":"2024-09-17T04:33:22.036154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:51.663061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036524+0000","last_change":"2024-09-17T04:33:19.686301+0000","last_active":"2024-09-17T04:33:22.036524+0000","last_peered":"2024-09-17T04:33:22.036524+0000","last_clean":"2024-09-17T04:33:22.036524+0000","last_became_active":"2024-09-17T04:33:19.684744+0000","last_became_peered":"2024-09-17T04:33:19.684744+0000","last_unstale":"2024-09-17T04:33:22.036524+0000","last_undegraded":"2024-09-17T04:33:22.036524+0000","last_fullsized":"2024-09-17T04:33:22.036524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:19:22.607029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.676940+0000","last_change":"2024-09-17T04:33:18.690509+0000","last_active":"2024-09-17T04:33:21.676940+0000","last_peered":"2024-09-17T04:33:21.676940+0000","last_clean":"2024-09-17T04:33:21.676940+0000","last_became_active":"2024-09-17T04:33:18.690113+0000","last_became_peered":"2024-09-17T04:33:18.690113+0000","last_unstale":"2024-09-17T04:33:21.676940+0000","last_undegraded":"2024-09-17T04:33:21.676940+0000","last_fullsized":"2024-09-17T04:33:21.676940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:14.180879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035866+0000","last_change":"2024-09-17T04:33:18.691233+0000","last_active":"2024-09-17T04:33:22.035866+0000","last_peered":"2024-09-17T04:33:22.035866+0000","last_clean":"2024-09-17T04:33:22.035866+0000","last_became_active":"2024-09-17T04:33:18.690790+0000","last_became_peered":"2024-09-17T04:33:18.690790+0000","last_unstale":"2024-09-17T04:33:22.035866+0000","last_undegraded":"2024-09-17T04:33:22.035866+0000","last_fullsized":"2024-09-17T04:33:22.035866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:14.180879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035122+0000","last_change":"2024-09-17T04:33:19.695559+0000","last_active":"2024-09-17T04:33:22.035122+0000","last_peered":"2024-09-17T04:33:22.035122+0000","last_clean":"2024-09-17T04:33:22.035122+0000","last_became_active":"2024-09-17T04:33:19.695236+0000","last_became_peered":"2024-09-17T04:33:19.695236+0000","last_unstale":"2024-09-17T04:33:22.035122+0000","last_undegraded":"2024-09-17T04:33:22.035122+0000","last_fullsized":"2024-09-17T04:33:22.035122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:19:59.773350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930110+0000","last_change":"2024-09-17T04:33:18.689567+0000","last_active":"2024-09-17T04:33:20.930110+0000","last_peered":"2024-09-17T04:33:20.930110+0000","last_clean":"2024-09-17T04:33:20.930110+0000","last_became_active":"2024-09-17T04:33:18.689202+0000","last_became_peered":"2024-09-17T04:33:18.689202+0000","last_unstale":"2024-09-17T04:33:20.930110+0000","last_undegraded":"2024-09-17T04:33:20.930110+0000","last_fullsized":"2024-09-17T04:33:20.930110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:23.445663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674031+0000","last_change":"2024-09-17T04:33:19.687275+0000","last_active":"2024-09-17T04:33:21.674031+0000","last_peered":"2024-09-17T04:33:21.674031+0000","last_clean":"2024-09-17T04:33:21.674031+0000","last_became_active":"2024-09-17T04:33:19.684741+0000","last_became_peered":"2024-09-17T04:33:19.684741+0000","last_unstale":"2024-09-17T04:33:21.674031+0000","last_undegraded":"2024-09-17T04:33:21.674031+0000","last_fullsized":"2024-09-17T04:33:21.674031+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:54:01.898864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677365+0000","last_change":"2024-09-17T04:33:18.694470+0000","last_active":"2024-09-17T04:33:21.677365+0000","last_peered":"2024-09-17T04:33:21.677365+0000","last_clean":"2024-09-17T04:33:21.677365+0000","last_became_active":"2024-09-17T04:33:18.694081+0000","last_became_peered":"2024-09-17T04:33:18.694081+0000","last_unstale":"2024-09-17T04:33:21.677365+0000","last_undegraded":"2024-09-17T04:33:21.677365+0000","last_fullsized":"2024-09-17T04:33:21.677365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:27.606246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930243+0000","last_change":"2024-09-17T04:33:19.695793+0000","last_active":"2024-09-17T04:33:20.930243+0000","last_peered":"2024-09-17T04:33:20.930243+0000","last_clean":"2024-09-17T04:33:20.930243+0000","last_became_active":"2024-09-17T04:33:19.695495+0000","last_became_peered":"2024-09-17T04:33:19.695495+0000","last_unstale":"2024-09-17T04:33:20.930243+0000","last_undegraded":"2024-09-17T04:33:20.930243+0000","last_fullsized":"2024-09-17T04:33:20.930243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:10:43.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930373+0000","last_change":"2024-09-17T04:33:18.694128+0000","last_active":"2024-09-17T04:33:20.930373+0000","last_peered":"2024-09-17T04:33:20.930373+0000","last_clean":"2024-09-17T04:33:20.930373+0000","last_became_active":"2024-09-17T04:33:18.693766+0000","last_became_peered":"2024-09-17T04:33:18.693766+0000","last_unstale":"2024-09-17T04:33:20.930373+0000","last_undegraded":"2024-09-17T04:33:20.930373+0000","last_fullsized":"2024-09-17T04:33:20.930373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:45:37.943381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036199+0000","last_change":"2024-09-17T04:33:19.691619+0000","last_active":"2024-09-17T04:33:22.036199+0000","last_peered":"2024-09-17T04:33:22.036199+0000","last_clean":"2024-09-17T04:33:22.036199+0000","last_became_active":"2024-09-17T04:33:19.690956+0000","last_became_peered":"2024-09-17T04:33:19.690956+0000","last_unstale":"2024-09-17T04:33:22.036199+0000","last_undegraded":"2024-09-17T04:33:22.036199+0000","last_fullsized":"2024-09-17T04:33:22.036199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:54:01.898864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674190+0000","last_change":"2024-09-17T04:33:18.691537+0000","last_active":"2024-09-17T04:33:21.674190+0000","last_peered":"2024-09-17T04:33:21.674190+0000","last_clean":"2024-09-17T04:33:21.674190+0000","last_became_active":"2024-09-17T04:33:18.691093+0000","last_became_peered":"2024-09-17T04:33:18.691093+0000","last_unstale":"2024-09-17T04:33:21.674190+0000","last_undegraded":"2024-09-17T04:33:21.674190+0000","last_fullsized":"2024-09-17T04:33:21.674190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:27.606246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691734+0000","last_change":"2024-09-17T04:33:19.691142+0000","last_active":"2024-09-17T04:33:20.691734+0000","last_peered":"2024-09-17T04:33:20.691734+0000","last_clean":"2024-09-17T04:33:20.691734+0000","last_became_active":"2024-09-17T04:33:19.690836+0000","last_became_peered":"2024-09-17T04:33:19.690836+0000","last_unstale":"2024-09-17T04:33:20.691734+0000","last_undegraded":"2024-09-17T04:33:20.691734+0000","last_fullsized":"2024-09-17T04:33:20.691734+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:36:40.026116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677278+0000","last_change":"2024-09-17T04:33:18.691686+0000","last_active":"2024-09-17T04:33:21.677278+0000","last_peered":"2024-09-17T04:33:21.677278+0000","last_clean":"2024-09-17T04:33:21.677278+0000","last_became_active":"2024-09-17T04:33:18.690643+0000","last_became_peered":"2024-09-17T04:33:18.690643+0000","last_unstale":"2024-09-17T04:33:21.677278+0000","last_undegraded":"2024-09-17T04:33:21.677278+0000","last_fullsized":"2024-09-17T04:33:21.677278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:53:28.012165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690245+0000","last_change":"2024-09-17T04:33:19.695036+0000","last_active":"2024-09-17T04:33:20.690245+0000","last_peered":"2024-09-17T04:33:20.690245+0000","last_clean":"2024-09-17T04:33:20.690245+0000","last_became_active":"2024-09-17T04:33:19.694670+0000","last_became_peered":"2024-09-17T04:33:19.694670+0000","last_unstale":"2024-09-17T04:33:20.690245+0000","last_undegraded":"2024-09-17T04:33:20.690245+0000","last_fullsized":"2024-09-17T04:33:20.690245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:44.199782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036569+0000","last_change":"2024-09-17T04:33:18.678408+0000","last_active":"2024-09-17T04:33:22.036569+0000","last_peered":"2024-09-17T04:33:22.036569+0000","last_clean":"2024-09-17T04:33:22.036569+0000","last_became_active":"2024-09-17T04:33:18.678054+0000","last_became_peered":"2024-09-17T04:33:18.678054+0000","last_unstale":"2024-09-17T04:33:22.036569+0000","last_undegraded":"2024-09-17T04:33:22.036569+0000","last_fullsized":"2024-09-17T04:33:22.036569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:51.663061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036608+0000","last_change":"2024-09-17T04:33:19.691672+0000","last_active":"2024-09-17T04:33:22.036608+0000","last_peered":"2024-09-17T04:33:22.036608+0000","last_clean":"2024-09-17T04:33:22.036608+0000","last_became_active":"2024-09-17T04:33:19.691319+0000","last_became_peered":"2024-09-17T04:33:19.691319+0000","last_unstale":"2024-09-17T04:33:22.036608+0000","last_undegraded":"2024-09-17T04:33:22.036608+0000","last_fullsized":"2024-09-17T04:33:22.036608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:44.305489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930312+0000","last_change":"2024-09-17T04:33:18.691730+0000","last_active":"2024-09-17T04:33:20.930312+0000","last_peered":"2024-09-17T04:33:20.930312+0000","last_clean":"2024-09-17T04:33:20.930312+0000","last_became_active":"2024-09-17T04:33:18.691046+0000","last_became_peered":"2024-09-17T04:33:18.691046+0000","last_unstale":"2024-09-17T04:33:20.930312+0000","last_undegraded":"2024-09-17T04:33:20.930312+0000","last_fullsized":"2024-09-17T04:33:20.930312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:53:28.012165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690746+0000","last_change":"2024-09-17T04:33:19.699326+0000","last_active":"2024-09-17T04:33:20.690746+0000","last_peered":"2024-09-17T04:33:20.690746+0000","last_clean":"2024-09-17T04:33:20.690746+0000","last_became_active":"2024-09-17T04:33:19.698549+0000","last_became_peered":"2024-09-17T04:33:19.698549+0000","last_unstale":"2024-09-17T04:33:20.690746+0000","last_undegraded":"2024-09-17T04:33:20.690746+0000","last_fullsized":"2024-09-17T04:33:20.690746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:28.772498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.676878+0000","last_change":"2024-09-17T04:33:18.691268+0000","last_active":"2024-09-17T04:33:21.676878+0000","last_peered":"2024-09-17T04:33:21.676878+0000","last_clean":"2024-09-17T04:33:21.676878+0000","last_became_active":"2024-09-17T04:33:18.690946+0000","last_became_peered":"2024-09-17T04:33:18.690946+0000","last_unstale":"2024-09-17T04:33:21.676878+0000","last_undegraded":"2024-09-17T04:33:21.676878+0000","last_fullsized":"2024-09-17T04:33:21.676878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:23.445663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677083+0000","last_change":"2024-09-17T04:33:19.695301+0000","last_active":"2024-09-17T04:33:21.677083+0000","last_peered":"2024-09-17T04:33:21.677083+0000","last_clean":"2024-09-17T04:33:21.677083+0000","last_became_active":"2024-09-17T04:33:19.694951+0000","last_became_peered":"2024-09-17T04:33:19.694951+0000","last_unstale":"2024-09-17T04:33:21.677083+0000","last_undegraded":"2024-09-17T04:33:21.677083+0000","last_fullsized":"2024-09-17T04:33:21.677083+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:22.696900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.683934+0000","last_change":"2024-09-17T04:33:18.684789+0000","last_active":"2024-09-17T04:33:20.683934+0000","last_peered":"2024-09-17T04:33:20.683934+0000","last_clean":"2024-09-17T04:33:20.683934+0000","last_became_active":"2024-09-17T04:33:18.684452+0000","last_became_peered":"2024-09-17T04:33:18.684452+0000","last_unstale":"2024-09-17T04:33:20.683934+0000","last_undegraded":"2024-09-17T04:33:20.683934+0000","last_fullsized":"2024-09-17T04:33:20.683934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:15:32.568929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690027+0000","last_change":"2024-09-17T04:33:19.699323+0000","last_active":"2024-09-17T04:33:20.690027+0000","last_peered":"2024-09-17T04:33:20.690027+0000","last_clean":"2024-09-17T04:33:20.690027+0000","last_became_active":"2024-09-17T04:33:19.698603+0000","last_became_peered":"2024-09-17T04:33:19.698603+0000","last_unstale":"2024-09-17T04:33:20.690027+0000","last_undegraded":"2024-09-17T04:33:20.690027+0000","last_fullsized":"2024-09-17T04:33:20.690027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:13.459991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691033+0000","last_change":"2024-09-17T04:33:18.681934+0000","last_active":"2024-09-17T04:33:20.691033+0000","last_peered":"2024-09-17T04:33:20.691033+0000","last_clean":"2024-09-17T04:33:20.691033+0000","last_became_active":"2024-09-17T04:33:18.681572+0000","last_became_peered":"2024-09-17T04:33:18.681572+0000","last_unstale":"2024-09-17T04:33:20.691033+0000","last_undegraded":"2024-09-17T04:33:20.691033+0000","last_fullsized":"2024-09-17T04:33:20.691033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:59.885891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691190+0000","last_change":"2024-09-17T04:33:19.699024+0000","last_active":"2024-09-17T04:33:20.691190+0000","last_peered":"2024-09-17T04:33:20.691190+0000","last_clean":"2024-09-17T04:33:20.691190+0000","last_became_active":"2024-09-17T04:33:19.698680+0000","last_became_peered":"2024-09-17T04:33:19.698680+0000","last_unstale":"2024-09-17T04:33:20.691190+0000","last_undegraded":"2024-09-17T04:33:20.691190+0000","last_fullsized":"2024-09-17T04:33:20.691190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:29:25.361534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.684435+0000","last_change":"2024-09-17T04:33:18.686803+0000","last_active":"2024-09-17T04:33:20.684435+0000","last_peered":"2024-09-17T04:33:20.684435+0000","last_clean":"2024-09-17T04:33:20.684435+0000","last_became_active":"2024-09-17T04:33:18.686293+0000","last_became_peered":"2024-09-17T04:33:18.686293+0000","last_unstale":"2024-09-17T04:33:20.684435+0000","last_undegraded":"2024-09-17T04:33:20.684435+0000","last_fullsized":"2024-09-17T04:33:20.684435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:07:39.351672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691823+0000","last_change":"2024-09-17T04:33:19.700000+0000","last_active":"2024-09-17T04:33:20.691823+0000","last_peered":"2024-09-17T04:33:20.691823+0000","last_clean":"2024-09-17T04:33:20.691823+0000","last_became_active":"2024-09-17T04:33:19.699695+0000","last_became_peered":"2024-09-17T04:33:19.699695+0000","last_unstale":"2024-09-17T04:33:20.691823+0000","last_undegraded":"2024-09-17T04:33:20.691823+0000","last_fullsized":"2024-09-17T04:33:20.691823+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:00.711952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674416+0000","last_change":"2024-09-17T04:33:18.684112+0000","last_active":"2024-09-17T04:33:21.674416+0000","last_peered":"2024-09-17T04:33:21.674416+0000","last_clean":"2024-09-17T04:33:21.674416+0000","last_became_active":"2024-09-17T04:33:18.683623+0000","last_became_peered":"2024-09-17T04:33:18.683623+0000","last_unstale":"2024-09-17T04:33:21.674416+0000","last_undegraded":"2024-09-17T04:33:21.674416+0000","last_fullsized":"2024-09-17T04:33:21.674416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:53.979348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930602+0000","last_change":"2024-09-17T04:33:19.688648+0000","last_active":"2024-09-17T04:33:20.930602+0000","last_peered":"2024-09-17T04:33:20.930602+0000","last_clean":"2024-09-17T04:33:20.930602+0000","last_became_active":"2024-09-17T04:33:19.688294+0000","last_became_peered":"2024-09-17T04:33:19.688294+0000","last_unstale":"2024-09-17T04:33:20.930602+0000","last_undegraded":"2024-09-17T04:33:20.930602+0000","last_fullsized":"2024-09-17T04:33:20.930602+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:26:38.002625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.684314+0000","last_change":"2024-09-17T04:33:18.690158+0000","last_active":"2024-09-17T04:33:20.684314+0000","last_peered":"2024-09-17T04:33:20.684314+0000","last_clean":"2024-09-17T04:33:20.684314+0000","last_became_active":"2024-09-17T04:33:18.689802+0000","last_became_peered":"2024-09-17T04:33:18.689802+0000","last_unstale":"2024-09-17T04:33:20.684314+0000","last_undegraded":"2024-09-17T04:33:20.684314+0000","last_fullsized":"2024-09-17T04:33:20.684314+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:21:44.098443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690295+0000","last_change":"2024-09-17T04:33:19.702563+0000","last_active":"2024-09-17T04:33:20.690295+0000","last_peered":"2024-09-17T04:33:20.690295+0000","last_clean":"2024-09-17T04:33:20.690295+0000","last_became_active":"2024-09-17T04:33:19.702241+0000","last_became_peered":"2024-09-17T04:33:19.702241+0000","last_unstale":"2024-09-17T04:33:20.690295+0000","last_undegraded":"2024-09-17T04:33:20.690295+0000","last_fullsized":"2024-09-17T04:33:20.690295+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:56:26.043321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690019+0000","last_change":"2024-09-17T04:33:09.978742+0000","last_active":"2024-09-17T04:33:20.690019+0000","last_peered":"2024-09-17T04:33:20.690019+0000","last_clean":"2024-09-17T04:33:20.690019+0000","last_became_active":"2024-09-17T04:33:09.977496+0000","last_became_peered":"2024-09-17T04:33:09.977496+0000","last_unstale":"2024-09-17T04:33:20.690019+0000","last_undegraded":"2024-09-17T04:33:20.690019+0000","last_fullsized":"2024-09-17T04:33:20.690019+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:08.930340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:08.930340+0000","last_clean_scrub_stamp":"2024-09-17T04:33:08.930340+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:21:26.566102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930309+0000","last_change":"2024-09-17T04:33:18.691736+0000","last_active":"2024-09-17T04:33:20.930309+0000","last_peered":"2024-09-17T04:33:20.930309+0000","last_clean":"2024-09-17T04:33:20.930309+0000","last_became_active":"2024-09-17T04:33:18.691410+0000","last_became_peered":"2024-09-17T04:33:18.691410+0000","last_unstale":"2024-09-17T04:33:20.930309+0000","last_undegraded":"2024-09-17T04:33:20.930309+0000","last_fullsized":"2024-09-17T04:33:20.930309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:14.180879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035352+0000","last_change":"2024-09-17T04:33:19.701468+0000","last_active":"2024-09-17T04:33:22.035352+0000","last_peered":"2024-09-17T04:33:22.035352+0000","last_clean":"2024-09-17T04:33:22.035352+0000","last_became_active":"2024-09-17T04:33:19.700764+0000","last_became_peered":"2024-09-17T04:33:19.700764+0000","last_unstale":"2024-09-17T04:33:22.035352+0000","last_undegraded":"2024-09-17T04:33:22.035352+0000","last_fullsized":"2024-09-17T04:33:22.035352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:16.814580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.689948+0000","last_change":"2024-09-17T04:33:18.687948+0000","last_active":"2024-09-17T04:33:20.689948+0000","last_peered":"2024-09-17T04:33:20.689948+0000","last_clean":"2024-09-17T04:33:20.689948+0000","last_became_active":"2024-09-17T04:33:18.687432+0000","last_became_peered":"2024-09-17T04:33:18.687432+0000","last_unstale":"2024-09-17T04:33:20.689948+0000","last_undegraded":"2024-09-17T04:33:20.689948+0000","last_fullsized":"2024-09-17T04:33:20.689948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:45:37.943381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035502+0000","last_change":"2024-09-17T04:33:19.687267+0000","last_active":"2024-09-17T04:33:22.035502+0000","last_peered":"2024-09-17T04:33:22.035502+0000","last_clean":"2024-09-17T04:33:22.035502+0000","last_became_active":"2024-09-17T04:33:19.684583+0000","last_became_peered":"2024-09-17T04:33:19.684583+0000","last_unstale":"2024-09-17T04:33:22.035502+0000","last_undegraded":"2024-09-17T04:33:22.035502+0000","last_fullsized":"2024-09-17T04:33:22.035502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:44.326971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.684040+0000","last_change":"2024-09-17T04:33:18.683812+0000","last_active":"2024-09-17T04:33:20.684040+0000","last_peered":"2024-09-17T04:33:20.684040+0000","last_clean":"2024-09-17T04:33:20.684040+0000","last_became_active":"2024-09-17T04:33:18.683253+0000","last_became_peered":"2024-09-17T04:33:18.683253+0000","last_unstale":"2024-09-17T04:33:20.684040+0000","last_undegraded":"2024-09-17T04:33:20.684040+0000","last_fullsized":"2024-09-17T04:33:20.684040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:00:10.198377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674050+0000","last_change":"2024-09-17T04:33:19.688660+0000","last_active":"2024-09-17T04:33:21.674050+0000","last_peered":"2024-09-17T04:33:21.674050+0000","last_clean":"2024-09-17T04:33:21.674050+0000","last_became_active":"2024-09-17T04:33:19.688336+0000","last_became_peered":"2024-09-17T04:33:19.688336+0000","last_unstale":"2024-09-17T04:33:21.674050+0000","last_undegraded":"2024-09-17T04:33:21.674050+0000","last_fullsized":"2024-09-17T04:33:21.674050+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:12.894886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930502+0000","last_change":"2024-09-17T04:33:18.688213+0000","last_active":"2024-09-17T04:33:20.930502+0000","last_peered":"2024-09-17T04:33:20.930502+0000","last_clean":"2024-09-17T04:33:20.930502+0000","last_became_active":"2024-09-17T04:33:18.687839+0000","last_became_peered":"2024-09-17T04:33:18.687839+0000","last_unstale":"2024-09-17T04:33:20.930502+0000","last_undegraded":"2024-09-17T04:33:20.930502+0000","last_fullsized":"2024-09-17T04:33:20.930502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:27.606246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690522+0000","last_change":"2024-09-17T04:33:19.695246+0000","last_active":"2024-09-17T04:33:20.690522+0000","last_peered":"2024-09-17T04:33:20.690522+0000","last_clean":"2024-09-17T04:33:20.690522+0000","last_became_active":"2024-09-17T04:33:19.694929+0000","last_became_peered":"2024-09-17T04:33:19.694929+0000","last_unstale":"2024-09-17T04:33:20.690522+0000","last_undegraded":"2024-09-17T04:33:20.690522+0000","last_fullsized":"2024-09-17T04:33:20.690522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:38:26.907899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674093+0000","last_change":"2024-09-17T04:33:18.682334+0000","last_active":"2024-09-17T04:33:21.674093+0000","last_peered":"2024-09-17T04:33:21.674093+0000","last_clean":"2024-09-17T04:33:21.674093+0000","last_became_active":"2024-09-17T04:33:18.681988+0000","last_became_peered":"2024-09-17T04:33:18.681988+0000","last_unstale":"2024-09-17T04:33:21.674093+0000","last_undegraded":"2024-09-17T04:33:21.674093+0000","last_fullsized":"2024-09-17T04:33:21.674093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:40.963166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930502+0000","last_change":"2024-09-17T04:33:19.694190+0000","last_active":"2024-09-17T04:33:20.930502+0000","last_peered":"2024-09-17T04:33:20.930502+0000","last_clean":"2024-09-17T04:33:20.930502+0000","last_became_active":"2024-09-17T04:33:19.691073+0000","last_became_peered":"2024-09-17T04:33:19.691073+0000","last_unstale":"2024-09-17T04:33:20.930502+0000","last_undegraded":"2024-09-17T04:33:20.930502+0000","last_fullsized":"2024-09-17T04:33:20.930502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:04:44.157056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690492+0000","last_change":"2024-09-17T04:33:18.687984+0000","last_active":"2024-09-17T04:33:20.690492+0000","last_peered":"2024-09-17T04:33:20.690492+0000","last_clean":"2024-09-17T04:33:20.690492+0000","last_became_active":"2024-09-17T04:33:18.687519+0000","last_became_peered":"2024-09-17T04:33:18.687519+0000","last_unstale":"2024-09-17T04:33:20.690492+0000","last_undegraded":"2024-09-17T04:33:20.690492+0000","last_fullsized":"2024-09-17T04:33:20.690492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:45.809989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674114+0000","last_change":"2024-09-17T04:33:19.688598+0000","last_active":"2024-09-17T04:33:21.674114+0000","last_peered":"2024-09-17T04:33:21.674114+0000","last_clean":"2024-09-17T04:33:21.674114+0000","last_became_active":"2024-09-17T04:33:19.688240+0000","last_became_peered":"2024-09-17T04:33:19.688240+0000","last_unstale":"2024-09-17T04:33:21.674114+0000","last_undegraded":"2024-09-17T04:33:21.674114+0000","last_fullsized":"2024-09-17T04:33:21.674114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:39.340136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.676358+0000","last_change":"2024-09-17T04:33:18.693000+0000","last_active":"2024-09-17T04:33:21.676358+0000","last_peered":"2024-09-17T04:33:21.676358+0000","last_clean":"2024-09-17T04:33:21.676358+0000","last_became_active":"2024-09-17T04:33:18.692677+0000","last_became_peered":"2024-09-17T04:33:18.692677+0000","last_unstale":"2024-09-17T04:33:21.676358+0000","last_undegraded":"2024-09-17T04:33:21.676358+0000","last_fullsized":"2024-09-17T04:33:21.676358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:45:37.943381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035886+0000","last_change":"2024-09-17T04:33:19.695017+0000","last_active":"2024-09-17T04:33:22.035886+0000","last_peered":"2024-09-17T04:33:22.035886+0000","last_clean":"2024-09-17T04:33:22.035886+0000","last_became_active":"2024-09-17T04:33:19.694621+0000","last_became_peered":"2024-09-17T04:33:19.694621+0000","last_unstale":"2024-09-17T04:33:22.035886+0000","last_undegraded":"2024-09-17T04:33:22.035886+0000","last_fullsized":"2024-09-17T04:33:22.035886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:38.326621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677397+0000","last_change":"2024-09-17T04:33:18.685926+0000","last_active":"2024-09-17T04:33:21.677397+0000","last_peered":"2024-09-17T04:33:21.677397+0000","last_clean":"2024-09-17T04:33:21.677397+0000","last_became_active":"2024-09-17T04:33:18.685598+0000","last_became_peered":"2024-09-17T04:33:18.685598+0000","last_unstale":"2024-09-17T04:33:21.677397+0000","last_undegraded":"2024-09-17T04:33:21.677397+0000","last_fullsized":"2024-09-17T04:33:21.677397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:45:37.943381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035362+0000","last_change":"2024-09-17T04:33:19.681836+0000","last_active":"2024-09-17T04:33:22.035362+0000","last_peered":"2024-09-17T04:33:22.035362+0000","last_clean":"2024-09-17T04:33:22.035362+0000","last_became_active":"2024-09-17T04:33:19.681085+0000","last_became_peered":"2024-09-17T04:33:19.681085+0000","last_unstale":"2024-09-17T04:33:22.035362+0000","last_undegraded":"2024-09-17T04:33:22.035362+0000","last_fullsized":"2024-09-17T04:33:22.035362+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:58.841388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.689553+0000","last_change":"2024-09-17T04:33:18.688337+0000","last_active":"2024-09-17T04:33:20.689553+0000","last_peered":"2024-09-17T04:33:20.689553+0000","last_clean":"2024-09-17T04:33:20.689553+0000","last_became_active":"2024-09-17T04:33:18.687331+0000","last_became_peered":"2024-09-17T04:33:18.687331+0000","last_unstale":"2024-09-17T04:33:20.689553+0000","last_undegraded":"2024-09-17T04:33:20.689553+0000","last_fullsized":"2024-09-17T04:33:20.689553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:27.606246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691952+0000","last_change":"2024-09-17T04:33:19.690673+0000","last_active":"2024-09-17T04:33:20.691952+0000","last_peered":"2024-09-17T04:33:20.691952+0000","last_clean":"2024-09-17T04:33:20.691952+0000","last_became_active":"2024-09-17T04:33:19.690356+0000","last_became_peered":"2024-09-17T04:33:19.690356+0000","last_unstale":"2024-09-17T04:33:20.691952+0000","last_undegraded":"2024-09-17T04:33:20.691952+0000","last_fullsized":"2024-09-17T04:33:20.691952+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:42.127433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674278+0000","last_change":"2024-09-17T04:33:18.686258+0000","last_active":"2024-09-17T04:33:21.674278+0000","last_peered":"2024-09-17T04:33:21.674278+0000","last_clean":"2024-09-17T04:33:21.674278+0000","last_became_active":"2024-09-17T04:33:18.685943+0000","last_became_peered":"2024-09-17T04:33:18.685943+0000","last_unstale":"2024-09-17T04:33:21.674278+0000","last_undegraded":"2024-09-17T04:33:21.674278+0000","last_fullsized":"2024-09-17T04:33:21.674278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:23.445663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035293+0000","last_change":"2024-09-17T04:33:19.684510+0000","last_active":"2024-09-17T04:33:22.035293+0000","last_peered":"2024-09-17T04:33:22.035293+0000","last_clean":"2024-09-17T04:33:22.035293+0000","last_became_active":"2024-09-17T04:33:19.683614+0000","last_became_peered":"2024-09-17T04:33:19.683614+0000","last_unstale":"2024-09-17T04:33:22.035293+0000","last_undegraded":"2024-09-17T04:33:22.035293+0000","last_fullsized":"2024-09-17T04:33:22.035293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:22.696900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674344+0000","last_change":"2024-09-17T04:33:18.684287+0000","last_active":"2024-09-17T04:33:21.674344+0000","last_peered":"2024-09-17T04:33:21.674344+0000","last_clean":"2024-09-17T04:33:21.674344+0000","last_became_active":"2024-09-17T04:33:18.683822+0000","last_became_peered":"2024-09-17T04:33:18.683822+0000","last_unstale":"2024-09-17T04:33:21.674344+0000","last_undegraded":"2024-09-17T04:33:21.674344+0000","last_fullsized":"2024-09-17T04:33:21.674344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:53:28.012165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036101+0000","last_change":"2024-09-17T04:33:19.699298+0000","last_active":"2024-09-17T04:33:22.036101+0000","last_peered":"2024-09-17T04:33:22.036101+0000","last_clean":"2024-09-17T04:33:22.036101+0000","last_became_active":"2024-09-17T04:33:19.698925+0000","last_became_peered":"2024-09-17T04:33:19.698925+0000","last_unstale":"2024-09-17T04:33:22.036101+0000","last_undegraded":"2024-09-17T04:33:22.036101+0000","last_fullsized":"2024-09-17T04:33:22.036101+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:13.459991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690689+0000","last_change":"2024-09-17T04:33:18.685941+0000","last_active":"2024-09-17T04:33:20.690689+0000","last_peered":"2024-09-17T04:33:20.690689+0000","last_clean":"2024-09-17T04:33:20.690689+0000","last_became_active":"2024-09-17T04:33:18.685484+0000","last_became_peered":"2024-09-17T04:33:18.685484+0000","last_unstale":"2024-09-17T04:33:20.690689+0000","last_undegraded":"2024-09-17T04:33:20.690689+0000","last_fullsized":"2024-09-17T04:33:20.690689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:40.963166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677177+0000","last_change":"2024-09-17T04:33:19.685032+0000","last_active":"2024-09-17T04:33:21.677177+0000","last_peered":"2024-09-17T04:33:21.677177+0000","last_clean":"2024-09-17T04:33:21.677177+0000","last_became_active":"2024-09-17T04:33:19.683930+0000","last_became_peered":"2024-09-17T04:33:19.683930+0000","last_unstale":"2024-09-17T04:33:21.677177+0000","last_undegraded":"2024-09-17T04:33:21.677177+0000","last_fullsized":"2024-09-17T04:33:21.677177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:16.814580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930467+0000","last_change":"2024-09-17T04:33:19.691762+0000","last_active":"2024-09-17T04:33:20.930467+0000","last_peered":"2024-09-17T04:33:20.930467+0000","last_clean":"2024-09-17T04:33:20.930467+0000","last_became_active":"2024-09-17T04:33:19.690765+0000","last_became_peered":"2024-09-17T04:33:19.690765+0000","last_unstale":"2024-09-17T04:33:20.930467+0000","last_undegraded":"2024-09-17T04:33:20.930467+0000","last_fullsized":"2024-09-17T04:33:20.930467+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:59.693472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.689803+0000","last_change":"2024-09-17T04:33:18.694221+0000","last_active":"2024-09-17T04:33:20.689803+0000","last_peered":"2024-09-17T04:33:20.689803+0000","last_clean":"2024-09-17T04:33:20.689803+0000","last_became_active":"2024-09-17T04:33:18.693906+0000","last_became_peered":"2024-09-17T04:33:18.693906+0000","last_unstale":"2024-09-17T04:33:20.689803+0000","last_undegraded":"2024-09-17T04:33:20.689803+0000","last_fullsized":"2024-09-17T04:33:20.689803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:53.979348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930455+0000","last_change":"2024-09-17T04:33:19.692952+0000","last_active":"2024-09-17T04:33:20.930455+0000","last_peered":"2024-09-17T04:33:20.930455+0000","last_clean":"2024-09-17T04:33:20.930455+0000","last_became_active":"2024-09-17T04:33:19.692612+0000","last_became_peered":"2024-09-17T04:33:19.692612+0000","last_unstale":"2024-09-17T04:33:20.930455+0000","last_undegraded":"2024-09-17T04:33:20.930455+0000","last_fullsized":"2024-09-17T04:33:20.930455+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:12:48.590039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690377+0000","last_change":"2024-09-17T04:33:18.692471+0000","last_active":"2024-09-17T04:33:20.690377+0000","last_peered":"2024-09-17T04:33:20.690377+0000","last_clean":"2024-09-17T04:33:20.690377+0000","last_became_active":"2024-09-17T04:33:18.692129+0000","last_became_peered":"2024-09-17T04:33:18.692129+0000","last_unstale":"2024-09-17T04:33:20.690377+0000","last_undegraded":"2024-09-17T04:33:20.690377+0000","last_fullsized":"2024-09-17T04:33:20.690377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:14:47.283001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035813+0000","last_change":"2024-09-17T04:33:19.691269+0000","last_active":"2024-09-17T04:33:22.035813+0000","last_peered":"2024-09-17T04:33:22.035813+0000","last_clean":"2024-09-17T04:33:22.035813+0000","last_became_active":"2024-09-17T04:33:19.690548+0000","last_became_peered":"2024-09-17T04:33:19.690548+0000","last_unstale":"2024-09-17T04:33:22.035813+0000","last_undegraded":"2024-09-17T04:33:22.035813+0000","last_fullsized":"2024-09-17T04:33:22.035813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:03.560173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036319+0000","last_change":"2024-09-17T04:33:18.689670+0000","last_active":"2024-09-17T04:33:22.036319+0000","last_peered":"2024-09-17T04:33:22.036319+0000","last_clean":"2024-09-17T04:33:22.036319+0000","last_became_active":"2024-09-17T04:33:18.689277+0000","last_became_peered":"2024-09-17T04:33:18.689277+0000","last_unstale":"2024-09-17T04:33:22.036319+0000","last_undegraded":"2024-09-17T04:33:22.036319+0000","last_fullsized":"2024-09-17T04:33:22.036319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:30.774768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036234+0000","last_change":"2024-09-17T04:33:19.686603+0000","last_active":"2024-09-17T04:33:22.036234+0000","last_peered":"2024-09-17T04:33:22.036234+0000","last_clean":"2024-09-17T04:33:22.036234+0000","last_became_active":"2024-09-17T04:33:19.686252+0000","last_became_peered":"2024-09-17T04:33:19.686252+0000","last_unstale":"2024-09-17T04:33:22.036234+0000","last_undegraded":"2024-09-17T04:33:22.036234+0000","last_fullsized":"2024-09-17T04:33:22.036234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:44.199782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035982+0000","last_change":"2024-09-17T04:33:18.683562+0000","last_active":"2024-09-17T04:33:22.035982+0000","last_peered":"2024-09-17T04:33:22.035982+0000","last_clean":"2024-09-17T04:33:22.035982+0000","last_became_active":"2024-09-17T04:33:18.683155+0000","last_became_peered":"2024-09-17T04:33:18.683155+0000","last_unstale":"2024-09-17T04:33:22.035982+0000","last_undegraded":"2024-09-17T04:33:22.035982+0000","last_fullsized":"2024-09-17T04:33:22.035982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:59:56.848238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":823296,"data_stored":669858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":80520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":78998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":237568,"data_stored":79034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":74220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":78908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":79068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":669914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:33:23.749 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:33:23.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.902+0000 7f4ef21eb640 1 -- 172.21.15.37:0/2288302185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec0cf720 msgr2=0x7f4eec1604d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:23.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.902+0000 7f4ef21eb640 1 --2- 172.21.15.37:0/2288302185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec0cf720 0x7f4eec1604d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4eec0b4390 tx=0x7f4ee002a4d0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.902+0000 7f4ef21eb640 1 -- 172.21.15.37:0/2288302185 shutdown_connections 2024-09-17T04:33:23.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.902+0000 7f4ef21eb640 1 --2- 172.21.15.37:0/2288302185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec0cf720 0x7f4eec1604d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.902+0000 7f4ef21eb640 1 --2- 172.21.15.37:0/2288302185 >> v2:172.21.15.92:3300/0 conn(0x7f4eec15d960 0x7f4eec0cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.902+0000 7f4ef21eb640 1 -- 172.21.15.37:0/2288302185 >> 172.21.15.37:0/2288302185 conn(0x7f4eec069120 msgr2=0x7f4eec069530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:23.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.902+0000 7f4ef21eb640 1 -- 172.21.15.37:0/2288302185 shutdown_connections 2024-09-17T04:33:23.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.902+0000 7f4ef21eb640 1 -- 172.21.15.37:0/2288302185 wait complete. 2024-09-17T04:33:23.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.904+0000 7f4ef21eb640 1 Processor -- start 2024-09-17T04:33:23.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.904+0000 7f4ef21eb640 1 -- start start 2024-09-17T04:33:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.906+0000 7f4ef21eb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4eec0cf720 0x7f4eec159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.906+0000 7f4ef21eb640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4eec15d960 0x7f4eec1575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.906+0000 7f4ef21eb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec160a10 0x7f4eec157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.906+0000 7f4ef21eb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4eec1573f0 con 0x7f4eec160a10 2024-09-17T04:33:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.906+0000 7f4ef21eb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4eec157270 con 0x7f4eec0cf720 2024-09-17T04:33:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.906+0000 7f4ef21eb640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f4eec0cd460 con 0x7f4eec15d960 2024-09-17T04:33:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.906+0000 7f4eebfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec160a10 0x7f4eec157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.906+0000 7f4eebfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec160a10 0x7f4eec157ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40750/0 (socket says 172.21.15.37:40750) 2024-09-17T04:33:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.906+0000 7f4eebfff640 1 -- 172.21.15.37:0/2572790687 learned_addr learned my addr 172.21.15.37:0/2572790687 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.906+0000 7f4eeb7fe640 1 --2- 172.21.15.37:0/2572790687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4eec0cf720 0x7f4eec159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.906+0000 7f4eeaffd640 1 --2- 172.21.15.37:0/2572790687 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4eec15d960 0x7f4eec1575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.906+0000 7f4eebfff640 1 -- 172.21.15.37:0/2572790687 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4eec15d960 msgr2=0x7f4eec1575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.906+0000 7f4eebfff640 1 --2- 172.21.15.37:0/2572790687 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4eec15d960 0x7f4eec1575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.906+0000 7f4eebfff640 1 -- 172.21.15.37:0/2572790687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4eec0cf720 msgr2=0x7f4eec159d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.906+0000 7f4eebfff640 1 --2- 172.21.15.37:0/2572790687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4eec0cf720 0x7f4eec159d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.906+0000 7f4eebfff640 1 -- 172.21.15.37:0/2572790687 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eec1e6700 con 0x7f4eec160a10 2024-09-17T04:33:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.906+0000 7f4eebfff640 1 --2- 172.21.15.37:0/2572790687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec160a10 0x7f4eec157ae0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400b9c0 tx=0x7f4ee400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.907+0000 7f4ed3fff640 1 -- 172.21.15.37:0/2572790687 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4019070 con 0x7f4eec160a10 2024-09-17T04:33:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.907+0000 7f4ed3fff640 1 -- 172.21.15.37:0/2572790687 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ee400cc30 con 0x7f4eec160a10 2024-09-17T04:33:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.907+0000 7f4ef21eb640 1 -- 172.21.15.37:0/2572790687 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eec1e6970 con 0x7f4eec160a10 2024-09-17T04:33:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.907+0000 7f4ed3fff640 1 -- 172.21.15.37:0/2572790687 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee40146a0 con 0x7f4eec160a10 2024-09-17T04:33:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.907+0000 7f4ef21eb640 1 -- 172.21.15.37:0/2572790687 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eec1e6f30 con 0x7f4eec160a10 2024-09-17T04:33:23.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.908+0000 7f4ef21eb640 1 -- 172.21.15.37:0/2572790687 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eec1604d0 con 0x7f4eec160a10 2024-09-17T04:33:23.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.908+0000 7f4ed3fff640 1 -- 172.21.15.37:0/2572790687 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4ee401c070 con 0x7f4eec160a10 2024-09-17T04:33:23.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.909+0000 7f4ed3fff640 1 --2- 172.21.15.37:0/2572790687 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4ed8078b90 0x7f4ed807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:23.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.909+0000 7f4ed3fff640 1 -- 172.21.15.37:0/2572790687 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f4ee409ab60 con 0x7f4eec160a10 2024-09-17T04:33:23.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.909+0000 7f4eeb7fe640 1 --2- 172.21.15.37:0/2572790687 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4ed8078b90 0x7f4ed807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:23.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.910+0000 7f4eeb7fe640 1 --2- 172.21.15.37:0/2572790687 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4ed8078b90 0x7f4ed807b0a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4edc0058c0 tx=0x7f4edc005f60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:23.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:23.913+0000 7f4ed3fff640 1 -- 172.21.15.37:0/2572790687 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4ee400ce20 con 0x7f4eec160a10 2024-09-17T04:33:24.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.096+0000 7f4ef21eb640 1 -- 172.21.15.37:0/2572790687 --> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4eec0b3a40 con 0x7f4ed8078b90 2024-09-17T04:33:24.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.101+0000 7f4ed3fff640 1 -- 172.21.15.37:0/2572790687 <== mgr.4100 v2:172.21.15.92:6800/864337835 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251934 (secure 0 0 0) 0x7f4eec0b3a40 con 0x7f4ed8078b90 2024-09-17T04:33:24.103 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:24.104 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-17T04:33:24.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.104+0000 7f4ef21eb640 1 -- 172.21.15.37:0/2572790687 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4ed8078b90 msgr2=0x7f4ed807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:24.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.104+0000 7f4ef21eb640 1 --2- 172.21.15.37:0/2572790687 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4ed8078b90 0x7f4ed807b0a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4edc0058c0 tx=0x7f4edc005f60 comp rx=0 tx=0).stop 2024-09-17T04:33:24.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.104+0000 7f4ef21eb640 1 -- 172.21.15.37:0/2572790687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec160a10 msgr2=0x7f4eec157ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:24.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.104+0000 7f4ef21eb640 1 --2- 172.21.15.37:0/2572790687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec160a10 0x7f4eec157ae0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400b9c0 tx=0x7f4ee400be80 comp rx=0 tx=0).stop 2024-09-17T04:33:24.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.104+0000 7f4ef21eb640 1 -- 172.21.15.37:0/2572790687 shutdown_connections 2024-09-17T04:33:24.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.104+0000 7f4ef21eb640 1 --2- 172.21.15.37:0/2572790687 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4ed8078b90 0x7f4ed807b0a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:24.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.104+0000 7f4ef21eb640 1 --2- 172.21.15.37:0/2572790687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec160a10 0x7f4eec157ae0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:24.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.104+0000 7f4ef21eb640 1 --2- 172.21.15.37:0/2572790687 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4eec15d960 0x7f4eec1575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:24.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.104+0000 7f4ef21eb640 1 --2- 172.21.15.37:0/2572790687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4eec0cf720 0x7f4eec159d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:24.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.104+0000 7f4ef21eb640 1 -- 172.21.15.37:0/2572790687 >> 172.21.15.37:0/2572790687 conn(0x7f4eec069120 msgr2=0x7f4eec15c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:24.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.104+0000 7f4ef21eb640 1 -- 172.21.15.37:0/2572790687 shutdown_connections 2024-09-17T04:33:24.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.104+0000 7f4ef21eb640 1 -- 172.21.15.37:0/2572790687 wait complete. 2024-09-17T04:33:24.115 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-17T04:33:22.445928+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":129,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219212,"kb_used_data":2972,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755508,"statfs":{"total":773094113280,"available":772869640192,"internally_reserved":0,"allocated":3043328,"data_stored":1810520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.786082"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691577+0000","last_change":"2024-09-17T04:33:19.693609+0000","last_active":"2024-09-17T04:33:20.691577+0000","last_peered":"2024-09-17T04:33:20.691577+0000","last_clean":"2024-09-17T04:33:20.691577+0000","last_became_active":"2024-09-17T04:33:19.693159+0000","last_became_peered":"2024-09-17T04:33:19.693159+0000","last_unstale":"2024-09-17T04:33:20.691577+0000","last_undegraded":"2024-09-17T04:33:20.691577+0000","last_fullsized":"2024-09-17T04:33:20.691577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:29.058321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674309+0000","last_change":"2024-09-17T04:33:19.686724+0000","last_active":"2024-09-17T04:33:21.674309+0000","last_peered":"2024-09-17T04:33:21.674309+0000","last_clean":"2024-09-17T04:33:21.674309+0000","last_became_active":"2024-09-17T04:33:19.685840+0000","last_became_peered":"2024-09-17T04:33:19.685840+0000","last_unstale":"2024-09-17T04:33:21.674309+0000","last_undegraded":"2024-09-17T04:33:21.674309+0000","last_fullsized":"2024-09-17T04:33:21.674309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:21.722762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035198+0000","last_change":"2024-09-17T04:33:19.688324+0000","last_active":"2024-09-17T04:33:22.035198+0000","last_peered":"2024-09-17T04:33:22.035198+0000","last_clean":"2024-09-17T04:33:22.035198+0000","last_became_active":"2024-09-17T04:33:19.687914+0000","last_became_peered":"2024-09-17T04:33:19.687914+0000","last_unstale":"2024-09-17T04:33:22.035198+0000","last_undegraded":"2024-09-17T04:33:22.035198+0000","last_fullsized":"2024-09-17T04:33:22.035198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:44.753208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930207+0000","last_change":"2024-09-17T04:33:19.694987+0000","last_active":"2024-09-17T04:33:20.930207+0000","last_peered":"2024-09-17T04:33:20.930207+0000","last_clean":"2024-09-17T04:33:20.930207+0000","last_became_active":"2024-09-17T04:33:19.694651+0000","last_became_peered":"2024-09-17T04:33:19.694651+0000","last_unstale":"2024-09-17T04:33:20.930207+0000","last_undegraded":"2024-09-17T04:33:20.930207+0000","last_fullsized":"2024-09-17T04:33:20.930207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:57:44.135139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690540+0000","last_change":"2024-09-17T04:33:19.693592+0000","last_active":"2024-09-17T04:33:20.690540+0000","last_peered":"2024-09-17T04:33:20.690540+0000","last_clean":"2024-09-17T04:33:20.690540+0000","last_became_active":"2024-09-17T04:33:19.690470+0000","last_became_peered":"2024-09-17T04:33:19.690470+0000","last_unstale":"2024-09-17T04:33:20.690540+0000","last_undegraded":"2024-09-17T04:33:20.690540+0000","last_fullsized":"2024-09-17T04:33:20.690540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:16:38.450837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.929991+0000","last_change":"2024-09-17T04:33:19.696808+0000","last_active":"2024-09-17T04:33:20.929991+0000","last_peered":"2024-09-17T04:33:20.929991+0000","last_clean":"2024-09-17T04:33:20.929991+0000","last_became_active":"2024-09-17T04:33:19.696523+0000","last_became_peered":"2024-09-17T04:33:19.696523+0000","last_unstale":"2024-09-17T04:33:20.929991+0000","last_undegraded":"2024-09-17T04:33:20.929991+0000","last_fullsized":"2024-09-17T04:33:20.929991+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:21.722762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.689753+0000","last_change":"2024-09-17T04:33:19.698772+0000","last_active":"2024-09-17T04:33:20.689753+0000","last_peered":"2024-09-17T04:33:20.689753+0000","last_clean":"2024-09-17T04:33:20.689753+0000","last_became_active":"2024-09-17T04:33:19.698190+0000","last_became_peered":"2024-09-17T04:33:19.698190+0000","last_unstale":"2024-09-17T04:33:20.689753+0000","last_undegraded":"2024-09-17T04:33:20.689753+0000","last_fullsized":"2024-09-17T04:33:20.689753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:20.825186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035054+0000","last_change":"2024-09-17T04:33:19.702702+0000","last_active":"2024-09-17T04:33:22.035054+0000","last_peered":"2024-09-17T04:33:22.035054+0000","last_clean":"2024-09-17T04:33:22.035054+0000","last_became_active":"2024-09-17T04:33:19.701450+0000","last_became_peered":"2024-09-17T04:33:19.701450+0000","last_unstale":"2024-09-17T04:33:22.035054+0000","last_undegraded":"2024-09-17T04:33:22.035054+0000","last_fullsized":"2024-09-17T04:33:22.035054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:27.889015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691647+0000","last_change":"2024-09-17T04:33:19.702493+0000","last_active":"2024-09-17T04:33:20.691647+0000","last_peered":"2024-09-17T04:33:20.691647+0000","last_clean":"2024-09-17T04:33:20.691647+0000","last_became_active":"2024-09-17T04:33:19.702105+0000","last_became_peered":"2024-09-17T04:33:19.702105+0000","last_unstale":"2024-09-17T04:33:20.691647+0000","last_undegraded":"2024-09-17T04:33:20.691647+0000","last_fullsized":"2024-09-17T04:33:20.691647+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:21.242559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677475+0000","last_change":"2024-09-17T04:33:19.691454+0000","last_active":"2024-09-17T04:33:21.677475+0000","last_peered":"2024-09-17T04:33:21.677475+0000","last_clean":"2024-09-17T04:33:21.677475+0000","last_became_active":"2024-09-17T04:33:19.689654+0000","last_became_peered":"2024-09-17T04:33:19.689654+0000","last_unstale":"2024-09-17T04:33:21.677475+0000","last_undegraded":"2024-09-17T04:33:21.677475+0000","last_fullsized":"2024-09-17T04:33:21.677475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:49.699088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677244+0000","last_change":"2024-09-17T04:33:19.687918+0000","last_active":"2024-09-17T04:33:21.677244+0000","last_peered":"2024-09-17T04:33:21.677244+0000","last_clean":"2024-09-17T04:33:21.677244+0000","last_became_active":"2024-09-17T04:33:19.687575+0000","last_became_peered":"2024-09-17T04:33:19.687575+0000","last_unstale":"2024-09-17T04:33:21.677244+0000","last_undegraded":"2024-09-17T04:33:21.677244+0000","last_fullsized":"2024-09-17T04:33:21.677244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:39.340136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691463+0000","last_change":"2024-09-17T04:33:19.687282+0000","last_active":"2024-09-17T04:33:20.691463+0000","last_peered":"2024-09-17T04:33:20.691463+0000","last_clean":"2024-09-17T04:33:20.691463+0000","last_became_active":"2024-09-17T04:33:19.686308+0000","last_became_peered":"2024-09-17T04:33:19.686308+0000","last_unstale":"2024-09-17T04:33:20.691463+0000","last_undegraded":"2024-09-17T04:33:20.691463+0000","last_fullsized":"2024-09-17T04:33:20.691463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:05.880115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036019+0000","last_change":"2024-09-17T04:33:19.685671+0000","last_active":"2024-09-17T04:33:22.036019+0000","last_peered":"2024-09-17T04:33:22.036019+0000","last_clean":"2024-09-17T04:33:22.036019+0000","last_became_active":"2024-09-17T04:33:19.685339+0000","last_became_peered":"2024-09-17T04:33:19.685339+0000","last_unstale":"2024-09-17T04:33:22.036019+0000","last_undegraded":"2024-09-17T04:33:22.036019+0000","last_fullsized":"2024-09-17T04:33:22.036019+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:20.825186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036606+0000","last_change":"2024-09-17T04:33:19.682161+0000","last_active":"2024-09-17T04:33:22.036606+0000","last_peered":"2024-09-17T04:33:22.036606+0000","last_clean":"2024-09-17T04:33:22.036606+0000","last_became_active":"2024-09-17T04:33:19.679848+0000","last_became_peered":"2024-09-17T04:33:19.679848+0000","last_unstale":"2024-09-17T04:33:22.036606+0000","last_undegraded":"2024-09-17T04:33:22.036606+0000","last_fullsized":"2024-09-17T04:33:22.036606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:49.699088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674099+0000","last_change":"2024-09-17T04:33:19.681771+0000","last_active":"2024-09-17T04:33:21.674099+0000","last_peered":"2024-09-17T04:33:21.674099+0000","last_clean":"2024-09-17T04:33:21.674099+0000","last_became_active":"2024-09-17T04:33:19.681448+0000","last_became_peered":"2024-09-17T04:33:19.681448+0000","last_unstale":"2024-09-17T04:33:21.674099+0000","last_undegraded":"2024-09-17T04:33:21.674099+0000","last_fullsized":"2024-09-17T04:33:21.674099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:19.224546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.929925+0000","last_change":"2024-09-17T04:33:19.701067+0000","last_active":"2024-09-17T04:33:20.929925+0000","last_peered":"2024-09-17T04:33:20.929925+0000","last_clean":"2024-09-17T04:33:20.929925+0000","last_became_active":"2024-09-17T04:33:19.698312+0000","last_became_peered":"2024-09-17T04:33:19.698312+0000","last_unstale":"2024-09-17T04:33:20.929925+0000","last_undegraded":"2024-09-17T04:33:20.929925+0000","last_fullsized":"2024-09-17T04:33:20.929925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:49.699088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930842+0000","last_change":"2024-09-17T04:33:19.698864+0000","last_active":"2024-09-17T04:33:20.930842+0000","last_peered":"2024-09-17T04:33:20.930842+0000","last_clean":"2024-09-17T04:33:20.930842+0000","last_became_active":"2024-09-17T04:33:19.698394+0000","last_became_peered":"2024-09-17T04:33:19.698394+0000","last_unstale":"2024-09-17T04:33:20.930842+0000","last_undegraded":"2024-09-17T04:33:20.930842+0000","last_fullsized":"2024-09-17T04:33:20.930842+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:16:06.475955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.684437+0000","last_change":"2024-09-17T04:33:19.692126+0000","last_active":"2024-09-17T04:33:20.684437+0000","last_peered":"2024-09-17T04:33:20.684437+0000","last_clean":"2024-09-17T04:33:20.684437+0000","last_became_active":"2024-09-17T04:33:19.691759+0000","last_became_peered":"2024-09-17T04:33:19.691759+0000","last_unstale":"2024-09-17T04:33:20.684437+0000","last_undegraded":"2024-09-17T04:33:20.684437+0000","last_fullsized":"2024-09-17T04:33:20.684437+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:48.205935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690446+0000","last_change":"2024-09-17T04:33:19.693246+0000","last_active":"2024-09-17T04:33:20.690446+0000","last_peered":"2024-09-17T04:33:20.690446+0000","last_clean":"2024-09-17T04:33:20.690446+0000","last_became_active":"2024-09-17T04:33:19.692591+0000","last_became_peered":"2024-09-17T04:33:19.692591+0000","last_unstale":"2024-09-17T04:33:20.690446+0000","last_undegraded":"2024-09-17T04:33:20.690446+0000","last_fullsized":"2024-09-17T04:33:20.690446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:58.709184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677180+0000","last_change":"2024-09-17T04:33:19.684484+0000","last_active":"2024-09-17T04:33:21.677180+0000","last_peered":"2024-09-17T04:33:21.677180+0000","last_clean":"2024-09-17T04:33:21.677180+0000","last_became_active":"2024-09-17T04:33:19.683795+0000","last_became_peered":"2024-09-17T04:33:19.683795+0000","last_unstale":"2024-09-17T04:33:21.677180+0000","last_undegraded":"2024-09-17T04:33:21.677180+0000","last_fullsized":"2024-09-17T04:33:21.677180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:03.560173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036302+0000","last_change":"2024-09-17T04:33:19.691536+0000","last_active":"2024-09-17T04:33:22.036302+0000","last_peered":"2024-09-17T04:33:22.036302+0000","last_clean":"2024-09-17T04:33:22.036302+0000","last_became_active":"2024-09-17T04:33:19.691142+0000","last_became_peered":"2024-09-17T04:33:19.691142+0000","last_unstale":"2024-09-17T04:33:22.036302+0000","last_undegraded":"2024-09-17T04:33:22.036302+0000","last_fullsized":"2024-09-17T04:33:22.036302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:12.894886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930139+0000","last_change":"2024-09-17T04:33:19.700787+0000","last_active":"2024-09-17T04:33:20.930139+0000","last_peered":"2024-09-17T04:33:20.930139+0000","last_clean":"2024-09-17T04:33:20.930139+0000","last_became_active":"2024-09-17T04:33:19.700430+0000","last_became_peered":"2024-09-17T04:33:19.700430+0000","last_unstale":"2024-09-17T04:33:20.930139+0000","last_undegraded":"2024-09-17T04:33:20.930139+0000","last_fullsized":"2024-09-17T04:33:20.930139+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:14.652736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035743+0000","last_change":"2024-09-17T04:33:19.684011+0000","last_active":"2024-09-17T04:33:22.035743+0000","last_peered":"2024-09-17T04:33:22.035743+0000","last_clean":"2024-09-17T04:33:22.035743+0000","last_became_active":"2024-09-17T04:33:19.683695+0000","last_became_peered":"2024-09-17T04:33:19.683695+0000","last_unstale":"2024-09-17T04:33:22.035743+0000","last_undegraded":"2024-09-17T04:33:22.035743+0000","last_fullsized":"2024-09-17T04:33:22.035743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:11.769729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930009+0000","last_change":"2024-09-17T04:33:19.689591+0000","last_active":"2024-09-17T04:33:20.930009+0000","last_peered":"2024-09-17T04:33:20.930009+0000","last_clean":"2024-09-17T04:33:20.930009+0000","last_became_active":"2024-09-17T04:33:19.689263+0000","last_became_peered":"2024-09-17T04:33:19.689263+0000","last_unstale":"2024-09-17T04:33:20.930009+0000","last_undegraded":"2024-09-17T04:33:20.930009+0000","last_fullsized":"2024-09-17T04:33:20.930009+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:06:04.595656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035649+0000","last_change":"2024-09-17T04:33:19.689915+0000","last_active":"2024-09-17T04:33:22.035649+0000","last_peered":"2024-09-17T04:33:22.035649+0000","last_clean":"2024-09-17T04:33:22.035649+0000","last_became_active":"2024-09-17T04:33:19.689612+0000","last_became_peered":"2024-09-17T04:33:19.689612+0000","last_unstale":"2024-09-17T04:33:22.035649+0000","last_undegraded":"2024-09-17T04:33:22.035649+0000","last_fullsized":"2024-09-17T04:33:22.035649+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:39.340136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677482+0000","last_change":"2024-09-17T04:33:19.695306+0000","last_active":"2024-09-17T04:33:21.677482+0000","last_peered":"2024-09-17T04:33:21.677482+0000","last_clean":"2024-09-17T04:33:21.677482+0000","last_became_active":"2024-09-17T04:33:19.694980+0000","last_became_peered":"2024-09-17T04:33:19.694980+0000","last_unstale":"2024-09-17T04:33:21.677482+0000","last_undegraded":"2024-09-17T04:33:21.677482+0000","last_fullsized":"2024-09-17T04:33:21.677482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:11.769729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674250+0000","last_change":"2024-09-17T04:33:19.684015+0000","last_active":"2024-09-17T04:33:21.674250+0000","last_peered":"2024-09-17T04:33:21.674250+0000","last_clean":"2024-09-17T04:33:21.674250+0000","last_became_active":"2024-09-17T04:33:19.683668+0000","last_became_peered":"2024-09-17T04:33:19.683668+0000","last_unstale":"2024-09-17T04:33:21.674250+0000","last_undegraded":"2024-09-17T04:33:21.674250+0000","last_fullsized":"2024-09-17T04:33:21.674250+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:49.699088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035804+0000","last_change":"2024-09-17T04:33:19.692206+0000","last_active":"2024-09-17T04:33:22.035804+0000","last_peered":"2024-09-17T04:33:22.035804+0000","last_clean":"2024-09-17T04:33:22.035804+0000","last_became_active":"2024-09-17T04:33:19.691739+0000","last_became_peered":"2024-09-17T04:33:19.691739+0000","last_unstale":"2024-09-17T04:33:22.035804+0000","last_undegraded":"2024-09-17T04:33:22.035804+0000","last_fullsized":"2024-09-17T04:33:22.035804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:44.305489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036443+0000","last_change":"2024-09-17T04:33:19.688554+0000","last_active":"2024-09-17T04:33:22.036443+0000","last_peered":"2024-09-17T04:33:22.036443+0000","last_clean":"2024-09-17T04:33:22.036443+0000","last_became_active":"2024-09-17T04:33:19.688216+0000","last_became_peered":"2024-09-17T04:33:19.688216+0000","last_unstale":"2024-09-17T04:33:22.036443+0000","last_undegraded":"2024-09-17T04:33:22.036443+0000","last_fullsized":"2024-09-17T04:33:22.036443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:19.224546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930414+0000","last_change":"2024-09-17T04:33:19.690117+0000","last_active":"2024-09-17T04:33:20.930414+0000","last_peered":"2024-09-17T04:33:20.930414+0000","last_clean":"2024-09-17T04:33:20.930414+0000","last_became_active":"2024-09-17T04:33:19.689494+0000","last_became_peered":"2024-09-17T04:33:19.689494+0000","last_unstale":"2024-09-17T04:33:20.930414+0000","last_undegraded":"2024-09-17T04:33:20.930414+0000","last_fullsized":"2024-09-17T04:33:20.930414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:12.356559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.684135+0000","last_change":"2024-09-17T04:33:19.702680+0000","last_active":"2024-09-17T04:33:20.684135+0000","last_peered":"2024-09-17T04:33:20.684135+0000","last_clean":"2024-09-17T04:33:20.684135+0000","last_became_active":"2024-09-17T04:33:19.702240+0000","last_became_peered":"2024-09-17T04:33:19.702240+0000","last_unstale":"2024-09-17T04:33:20.684135+0000","last_undegraded":"2024-09-17T04:33:20.684135+0000","last_fullsized":"2024-09-17T04:33:20.684135+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:56:27.523263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677072+0000","last_change":"2024-09-17T04:33:19.698110+0000","last_active":"2024-09-17T04:33:21.677072+0000","last_peered":"2024-09-17T04:33:21.677072+0000","last_clean":"2024-09-17T04:33:21.677072+0000","last_became_active":"2024-09-17T04:33:19.697800+0000","last_became_peered":"2024-09-17T04:33:19.697800+0000","last_unstale":"2024-09-17T04:33:21.677072+0000","last_undegraded":"2024-09-17T04:33:21.677072+0000","last_fullsized":"2024-09-17T04:33:21.677072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:27.889015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.689882+0000","last_change":"2024-09-17T04:33:19.682605+0000","last_active":"2024-09-17T04:33:20.689882+0000","last_peered":"2024-09-17T04:33:20.689882+0000","last_clean":"2024-09-17T04:33:20.689882+0000","last_became_active":"2024-09-17T04:33:19.682212+0000","last_became_peered":"2024-09-17T04:33:19.682212+0000","last_unstale":"2024-09-17T04:33:20.689882+0000","last_undegraded":"2024-09-17T04:33:20.689882+0000","last_fullsized":"2024-09-17T04:33:20.689882+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:36:00.513397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036154+0000","last_change":"2024-09-17T04:33:18.690638+0000","last_active":"2024-09-17T04:33:22.036154+0000","last_peered":"2024-09-17T04:33:22.036154+0000","last_clean":"2024-09-17T04:33:22.036154+0000","last_became_active":"2024-09-17T04:33:18.690326+0000","last_became_peered":"2024-09-17T04:33:18.690326+0000","last_unstale":"2024-09-17T04:33:22.036154+0000","last_undegraded":"2024-09-17T04:33:22.036154+0000","last_fullsized":"2024-09-17T04:33:22.036154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:51.663061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036524+0000","last_change":"2024-09-17T04:33:19.686301+0000","last_active":"2024-09-17T04:33:22.036524+0000","last_peered":"2024-09-17T04:33:22.036524+0000","last_clean":"2024-09-17T04:33:22.036524+0000","last_became_active":"2024-09-17T04:33:19.684744+0000","last_became_peered":"2024-09-17T04:33:19.684744+0000","last_unstale":"2024-09-17T04:33:22.036524+0000","last_undegraded":"2024-09-17T04:33:22.036524+0000","last_fullsized":"2024-09-17T04:33:22.036524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:19:22.607029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.676940+0000","last_change":"2024-09-17T04:33:18.690509+0000","last_active":"2024-09-17T04:33:21.676940+0000","last_peered":"2024-09-17T04:33:21.676940+0000","last_clean":"2024-09-17T04:33:21.676940+0000","last_became_active":"2024-09-17T04:33:18.690113+0000","last_became_peered":"2024-09-17T04:33:18.690113+0000","last_unstale":"2024-09-17T04:33:21.676940+0000","last_undegraded":"2024-09-17T04:33:21.676940+0000","last_fullsized":"2024-09-17T04:33:21.676940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:14.180879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035866+0000","last_change":"2024-09-17T04:33:18.691233+0000","last_active":"2024-09-17T04:33:22.035866+0000","last_peered":"2024-09-17T04:33:22.035866+0000","last_clean":"2024-09-17T04:33:22.035866+0000","last_became_active":"2024-09-17T04:33:18.690790+0000","last_became_peered":"2024-09-17T04:33:18.690790+0000","last_unstale":"2024-09-17T04:33:22.035866+0000","last_undegraded":"2024-09-17T04:33:22.035866+0000","last_fullsized":"2024-09-17T04:33:22.035866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:14.180879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035122+0000","last_change":"2024-09-17T04:33:19.695559+0000","last_active":"2024-09-17T04:33:22.035122+0000","last_peered":"2024-09-17T04:33:22.035122+0000","last_clean":"2024-09-17T04:33:22.035122+0000","last_became_active":"2024-09-17T04:33:19.695236+0000","last_became_peered":"2024-09-17T04:33:19.695236+0000","last_unstale":"2024-09-17T04:33:22.035122+0000","last_undegraded":"2024-09-17T04:33:22.035122+0000","last_fullsized":"2024-09-17T04:33:22.035122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:19:59.773350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930110+0000","last_change":"2024-09-17T04:33:18.689567+0000","last_active":"2024-09-17T04:33:20.930110+0000","last_peered":"2024-09-17T04:33:20.930110+0000","last_clean":"2024-09-17T04:33:20.930110+0000","last_became_active":"2024-09-17T04:33:18.689202+0000","last_became_peered":"2024-09-17T04:33:18.689202+0000","last_unstale":"2024-09-17T04:33:20.930110+0000","last_undegraded":"2024-09-17T04:33:20.930110+0000","last_fullsized":"2024-09-17T04:33:20.930110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:23.445663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674031+0000","last_change":"2024-09-17T04:33:19.687275+0000","last_active":"2024-09-17T04:33:21.674031+0000","last_peered":"2024-09-17T04:33:21.674031+0000","last_clean":"2024-09-17T04:33:21.674031+0000","last_became_active":"2024-09-17T04:33:19.684741+0000","last_became_peered":"2024-09-17T04:33:19.684741+0000","last_unstale":"2024-09-17T04:33:21.674031+0000","last_undegraded":"2024-09-17T04:33:21.674031+0000","last_fullsized":"2024-09-17T04:33:21.674031+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:54:01.898864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677365+0000","last_change":"2024-09-17T04:33:18.694470+0000","last_active":"2024-09-17T04:33:21.677365+0000","last_peered":"2024-09-17T04:33:21.677365+0000","last_clean":"2024-09-17T04:33:21.677365+0000","last_became_active":"2024-09-17T04:33:18.694081+0000","last_became_peered":"2024-09-17T04:33:18.694081+0000","last_unstale":"2024-09-17T04:33:21.677365+0000","last_undegraded":"2024-09-17T04:33:21.677365+0000","last_fullsized":"2024-09-17T04:33:21.677365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:27.606246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930243+0000","last_change":"2024-09-17T04:33:19.695793+0000","last_active":"2024-09-17T04:33:20.930243+0000","last_peered":"2024-09-17T04:33:20.930243+0000","last_clean":"2024-09-17T04:33:20.930243+0000","last_became_active":"2024-09-17T04:33:19.695495+0000","last_became_peered":"2024-09-17T04:33:19.695495+0000","last_unstale":"2024-09-17T04:33:20.930243+0000","last_undegraded":"2024-09-17T04:33:20.930243+0000","last_fullsized":"2024-09-17T04:33:20.930243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:10:43.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930373+0000","last_change":"2024-09-17T04:33:18.694128+0000","last_active":"2024-09-17T04:33:20.930373+0000","last_peered":"2024-09-17T04:33:20.930373+0000","last_clean":"2024-09-17T04:33:20.930373+0000","last_became_active":"2024-09-17T04:33:18.693766+0000","last_became_peered":"2024-09-17T04:33:18.693766+0000","last_unstale":"2024-09-17T04:33:20.930373+0000","last_undegraded":"2024-09-17T04:33:20.930373+0000","last_fullsized":"2024-09-17T04:33:20.930373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:45:37.943381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036199+0000","last_change":"2024-09-17T04:33:19.691619+0000","last_active":"2024-09-17T04:33:22.036199+0000","last_peered":"2024-09-17T04:33:22.036199+0000","last_clean":"2024-09-17T04:33:22.036199+0000","last_became_active":"2024-09-17T04:33:19.690956+0000","last_became_peered":"2024-09-17T04:33:19.690956+0000","last_unstale":"2024-09-17T04:33:22.036199+0000","last_undegraded":"2024-09-17T04:33:22.036199+0000","last_fullsized":"2024-09-17T04:33:22.036199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:54:01.898864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674190+0000","last_change":"2024-09-17T04:33:18.691537+0000","last_active":"2024-09-17T04:33:21.674190+0000","last_peered":"2024-09-17T04:33:21.674190+0000","last_clean":"2024-09-17T04:33:21.674190+0000","last_became_active":"2024-09-17T04:33:18.691093+0000","last_became_peered":"2024-09-17T04:33:18.691093+0000","last_unstale":"2024-09-17T04:33:21.674190+0000","last_undegraded":"2024-09-17T04:33:21.674190+0000","last_fullsized":"2024-09-17T04:33:21.674190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:27.606246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691734+0000","last_change":"2024-09-17T04:33:19.691142+0000","last_active":"2024-09-17T04:33:20.691734+0000","last_peered":"2024-09-17T04:33:20.691734+0000","last_clean":"2024-09-17T04:33:20.691734+0000","last_became_active":"2024-09-17T04:33:19.690836+0000","last_became_peered":"2024-09-17T04:33:19.690836+0000","last_unstale":"2024-09-17T04:33:20.691734+0000","last_undegraded":"2024-09-17T04:33:20.691734+0000","last_fullsized":"2024-09-17T04:33:20.691734+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:36:40.026116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677278+0000","last_change":"2024-09-17T04:33:18.691686+0000","last_active":"2024-09-17T04:33:21.677278+0000","last_peered":"2024-09-17T04:33:21.677278+0000","last_clean":"2024-09-17T04:33:21.677278+0000","last_became_active":"2024-09-17T04:33:18.690643+0000","last_became_peered":"2024-09-17T04:33:18.690643+0000","last_unstale":"2024-09-17T04:33:21.677278+0000","last_undegraded":"2024-09-17T04:33:21.677278+0000","last_fullsized":"2024-09-17T04:33:21.677278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:53:28.012165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690245+0000","last_change":"2024-09-17T04:33:19.695036+0000","last_active":"2024-09-17T04:33:20.690245+0000","last_peered":"2024-09-17T04:33:20.690245+0000","last_clean":"2024-09-17T04:33:20.690245+0000","last_became_active":"2024-09-17T04:33:19.694670+0000","last_became_peered":"2024-09-17T04:33:19.694670+0000","last_unstale":"2024-09-17T04:33:20.690245+0000","last_undegraded":"2024-09-17T04:33:20.690245+0000","last_fullsized":"2024-09-17T04:33:20.690245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:44.199782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036569+0000","last_change":"2024-09-17T04:33:18.678408+0000","last_active":"2024-09-17T04:33:22.036569+0000","last_peered":"2024-09-17T04:33:22.036569+0000","last_clean":"2024-09-17T04:33:22.036569+0000","last_became_active":"2024-09-17T04:33:18.678054+0000","last_became_peered":"2024-09-17T04:33:18.678054+0000","last_unstale":"2024-09-17T04:33:22.036569+0000","last_undegraded":"2024-09-17T04:33:22.036569+0000","last_fullsized":"2024-09-17T04:33:22.036569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:51.663061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036608+0000","last_change":"2024-09-17T04:33:19.691672+0000","last_active":"2024-09-17T04:33:22.036608+0000","last_peered":"2024-09-17T04:33:22.036608+0000","last_clean":"2024-09-17T04:33:22.036608+0000","last_became_active":"2024-09-17T04:33:19.691319+0000","last_became_peered":"2024-09-17T04:33:19.691319+0000","last_unstale":"2024-09-17T04:33:22.036608+0000","last_undegraded":"2024-09-17T04:33:22.036608+0000","last_fullsized":"2024-09-17T04:33:22.036608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:44.305489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930312+0000","last_change":"2024-09-17T04:33:18.691730+0000","last_active":"2024-09-17T04:33:20.930312+0000","last_peered":"2024-09-17T04:33:20.930312+0000","last_clean":"2024-09-17T04:33:20.930312+0000","last_became_active":"2024-09-17T04:33:18.691046+0000","last_became_peered":"2024-09-17T04:33:18.691046+0000","last_unstale":"2024-09-17T04:33:20.930312+0000","last_undegraded":"2024-09-17T04:33:20.930312+0000","last_fullsized":"2024-09-17T04:33:20.930312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:53:28.012165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690746+0000","last_change":"2024-09-17T04:33:19.699326+0000","last_active":"2024-09-17T04:33:20.690746+0000","last_peered":"2024-09-17T04:33:20.690746+0000","last_clean":"2024-09-17T04:33:20.690746+0000","last_became_active":"2024-09-17T04:33:19.698549+0000","last_became_peered":"2024-09-17T04:33:19.698549+0000","last_unstale":"2024-09-17T04:33:20.690746+0000","last_undegraded":"2024-09-17T04:33:20.690746+0000","last_fullsized":"2024-09-17T04:33:20.690746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:28.772498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.676878+0000","last_change":"2024-09-17T04:33:18.691268+0000","last_active":"2024-09-17T04:33:21.676878+0000","last_peered":"2024-09-17T04:33:21.676878+0000","last_clean":"2024-09-17T04:33:21.676878+0000","last_became_active":"2024-09-17T04:33:18.690946+0000","last_became_peered":"2024-09-17T04:33:18.690946+0000","last_unstale":"2024-09-17T04:33:21.676878+0000","last_undegraded":"2024-09-17T04:33:21.676878+0000","last_fullsized":"2024-09-17T04:33:21.676878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:23.445663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677083+0000","last_change":"2024-09-17T04:33:19.695301+0000","last_active":"2024-09-17T04:33:21.677083+0000","last_peered":"2024-09-17T04:33:21.677083+0000","last_clean":"2024-09-17T04:33:21.677083+0000","last_became_active":"2024-09-17T04:33:19.694951+0000","last_became_peered":"2024-09-17T04:33:19.694951+0000","last_unstale":"2024-09-17T04:33:21.677083+0000","last_undegraded":"2024-09-17T04:33:21.677083+0000","last_fullsized":"2024-09-17T04:33:21.677083+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:22.696900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.683934+0000","last_change":"2024-09-17T04:33:18.684789+0000","last_active":"2024-09-17T04:33:20.683934+0000","last_peered":"2024-09-17T04:33:20.683934+0000","last_clean":"2024-09-17T04:33:20.683934+0000","last_became_active":"2024-09-17T04:33:18.684452+0000","last_became_peered":"2024-09-17T04:33:18.684452+0000","last_unstale":"2024-09-17T04:33:20.683934+0000","last_undegraded":"2024-09-17T04:33:20.683934+0000","last_fullsized":"2024-09-17T04:33:20.683934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:15:32.568929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690027+0000","last_change":"2024-09-17T04:33:19.699323+0000","last_active":"2024-09-17T04:33:20.690027+0000","last_peered":"2024-09-17T04:33:20.690027+0000","last_clean":"2024-09-17T04:33:20.690027+0000","last_became_active":"2024-09-17T04:33:19.698603+0000","last_became_peered":"2024-09-17T04:33:19.698603+0000","last_unstale":"2024-09-17T04:33:20.690027+0000","last_undegraded":"2024-09-17T04:33:20.690027+0000","last_fullsized":"2024-09-17T04:33:20.690027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:13.459991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691033+0000","last_change":"2024-09-17T04:33:18.681934+0000","last_active":"2024-09-17T04:33:20.691033+0000","last_peered":"2024-09-17T04:33:20.691033+0000","last_clean":"2024-09-17T04:33:20.691033+0000","last_became_active":"2024-09-17T04:33:18.681572+0000","last_became_peered":"2024-09-17T04:33:18.681572+0000","last_unstale":"2024-09-17T04:33:20.691033+0000","last_undegraded":"2024-09-17T04:33:20.691033+0000","last_fullsized":"2024-09-17T04:33:20.691033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:59.885891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691190+0000","last_change":"2024-09-17T04:33:19.699024+0000","last_active":"2024-09-17T04:33:20.691190+0000","last_peered":"2024-09-17T04:33:20.691190+0000","last_clean":"2024-09-17T04:33:20.691190+0000","last_became_active":"2024-09-17T04:33:19.698680+0000","last_became_peered":"2024-09-17T04:33:19.698680+0000","last_unstale":"2024-09-17T04:33:20.691190+0000","last_undegraded":"2024-09-17T04:33:20.691190+0000","last_fullsized":"2024-09-17T04:33:20.691190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:29:25.361534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.684435+0000","last_change":"2024-09-17T04:33:18.686803+0000","last_active":"2024-09-17T04:33:20.684435+0000","last_peered":"2024-09-17T04:33:20.684435+0000","last_clean":"2024-09-17T04:33:20.684435+0000","last_became_active":"2024-09-17T04:33:18.686293+0000","last_became_peered":"2024-09-17T04:33:18.686293+0000","last_unstale":"2024-09-17T04:33:20.684435+0000","last_undegraded":"2024-09-17T04:33:20.684435+0000","last_fullsized":"2024-09-17T04:33:20.684435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:07:39.351672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691823+0000","last_change":"2024-09-17T04:33:19.700000+0000","last_active":"2024-09-17T04:33:20.691823+0000","last_peered":"2024-09-17T04:33:20.691823+0000","last_clean":"2024-09-17T04:33:20.691823+0000","last_became_active":"2024-09-17T04:33:19.699695+0000","last_became_peered":"2024-09-17T04:33:19.699695+0000","last_unstale":"2024-09-17T04:33:20.691823+0000","last_undegraded":"2024-09-17T04:33:20.691823+0000","last_fullsized":"2024-09-17T04:33:20.691823+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:00.711952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674416+0000","last_change":"2024-09-17T04:33:18.684112+0000","last_active":"2024-09-17T04:33:21.674416+0000","last_peered":"2024-09-17T04:33:21.674416+0000","last_clean":"2024-09-17T04:33:21.674416+0000","last_became_active":"2024-09-17T04:33:18.683623+0000","last_became_peered":"2024-09-17T04:33:18.683623+0000","last_unstale":"2024-09-17T04:33:21.674416+0000","last_undegraded":"2024-09-17T04:33:21.674416+0000","last_fullsized":"2024-09-17T04:33:21.674416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:53.979348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930602+0000","last_change":"2024-09-17T04:33:19.688648+0000","last_active":"2024-09-17T04:33:20.930602+0000","last_peered":"2024-09-17T04:33:20.930602+0000","last_clean":"2024-09-17T04:33:20.930602+0000","last_became_active":"2024-09-17T04:33:19.688294+0000","last_became_peered":"2024-09-17T04:33:19.688294+0000","last_unstale":"2024-09-17T04:33:20.930602+0000","last_undegraded":"2024-09-17T04:33:20.930602+0000","last_fullsized":"2024-09-17T04:33:20.930602+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:26:38.002625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.684314+0000","last_change":"2024-09-17T04:33:18.690158+0000","last_active":"2024-09-17T04:33:20.684314+0000","last_peered":"2024-09-17T04:33:20.684314+0000","last_clean":"2024-09-17T04:33:20.684314+0000","last_became_active":"2024-09-17T04:33:18.689802+0000","last_became_peered":"2024-09-17T04:33:18.689802+0000","last_unstale":"2024-09-17T04:33:20.684314+0000","last_undegraded":"2024-09-17T04:33:20.684314+0000","last_fullsized":"2024-09-17T04:33:20.684314+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:21:44.098443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690295+0000","last_change":"2024-09-17T04:33:19.702563+0000","last_active":"2024-09-17T04:33:20.690295+0000","last_peered":"2024-09-17T04:33:20.690295+0000","last_clean":"2024-09-17T04:33:20.690295+0000","last_became_active":"2024-09-17T04:33:19.702241+0000","last_became_peered":"2024-09-17T04:33:19.702241+0000","last_unstale":"2024-09-17T04:33:20.690295+0000","last_undegraded":"2024-09-17T04:33:20.690295+0000","last_fullsized":"2024-09-17T04:33:20.690295+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:56:26.043321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690019+0000","last_change":"2024-09-17T04:33:09.978742+0000","last_active":"2024-09-17T04:33:20.690019+0000","last_peered":"2024-09-17T04:33:20.690019+0000","last_clean":"2024-09-17T04:33:20.690019+0000","last_became_active":"2024-09-17T04:33:09.977496+0000","last_became_peered":"2024-09-17T04:33:09.977496+0000","last_unstale":"2024-09-17T04:33:20.690019+0000","last_undegraded":"2024-09-17T04:33:20.690019+0000","last_fullsized":"2024-09-17T04:33:20.690019+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:08.930340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:08.930340+0000","last_clean_scrub_stamp":"2024-09-17T04:33:08.930340+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:21:26.566102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930309+0000","last_change":"2024-09-17T04:33:18.691736+0000","last_active":"2024-09-17T04:33:20.930309+0000","last_peered":"2024-09-17T04:33:20.930309+0000","last_clean":"2024-09-17T04:33:20.930309+0000","last_became_active":"2024-09-17T04:33:18.691410+0000","last_became_peered":"2024-09-17T04:33:18.691410+0000","last_unstale":"2024-09-17T04:33:20.930309+0000","last_undegraded":"2024-09-17T04:33:20.930309+0000","last_fullsized":"2024-09-17T04:33:20.930309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:14.180879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035352+0000","last_change":"2024-09-17T04:33:19.701468+0000","last_active":"2024-09-17T04:33:22.035352+0000","last_peered":"2024-09-17T04:33:22.035352+0000","last_clean":"2024-09-17T04:33:22.035352+0000","last_became_active":"2024-09-17T04:33:19.700764+0000","last_became_peered":"2024-09-17T04:33:19.700764+0000","last_unstale":"2024-09-17T04:33:22.035352+0000","last_undegraded":"2024-09-17T04:33:22.035352+0000","last_fullsized":"2024-09-17T04:33:22.035352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:16.814580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.689948+0000","last_change":"2024-09-17T04:33:18.687948+0000","last_active":"2024-09-17T04:33:20.689948+0000","last_peered":"2024-09-17T04:33:20.689948+0000","last_clean":"2024-09-17T04:33:20.689948+0000","last_became_active":"2024-09-17T04:33:18.687432+0000","last_became_peered":"2024-09-17T04:33:18.687432+0000","last_unstale":"2024-09-17T04:33:20.689948+0000","last_undegraded":"2024-09-17T04:33:20.689948+0000","last_fullsized":"2024-09-17T04:33:20.689948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:45:37.943381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035502+0000","last_change":"2024-09-17T04:33:19.687267+0000","last_active":"2024-09-17T04:33:22.035502+0000","last_peered":"2024-09-17T04:33:22.035502+0000","last_clean":"2024-09-17T04:33:22.035502+0000","last_became_active":"2024-09-17T04:33:19.684583+0000","last_became_peered":"2024-09-17T04:33:19.684583+0000","last_unstale":"2024-09-17T04:33:22.035502+0000","last_undegraded":"2024-09-17T04:33:22.035502+0000","last_fullsized":"2024-09-17T04:33:22.035502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:44.326971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.684040+0000","last_change":"2024-09-17T04:33:18.683812+0000","last_active":"2024-09-17T04:33:20.684040+0000","last_peered":"2024-09-17T04:33:20.684040+0000","last_clean":"2024-09-17T04:33:20.684040+0000","last_became_active":"2024-09-17T04:33:18.683253+0000","last_became_peered":"2024-09-17T04:33:18.683253+0000","last_unstale":"2024-09-17T04:33:20.684040+0000","last_undegraded":"2024-09-17T04:33:20.684040+0000","last_fullsized":"2024-09-17T04:33:20.684040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:00:10.198377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674050+0000","last_change":"2024-09-17T04:33:19.688660+0000","last_active":"2024-09-17T04:33:21.674050+0000","last_peered":"2024-09-17T04:33:21.674050+0000","last_clean":"2024-09-17T04:33:21.674050+0000","last_became_active":"2024-09-17T04:33:19.688336+0000","last_became_peered":"2024-09-17T04:33:19.688336+0000","last_unstale":"2024-09-17T04:33:21.674050+0000","last_undegraded":"2024-09-17T04:33:21.674050+0000","last_fullsized":"2024-09-17T04:33:21.674050+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:12.894886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930502+0000","last_change":"2024-09-17T04:33:18.688213+0000","last_active":"2024-09-17T04:33:20.930502+0000","last_peered":"2024-09-17T04:33:20.930502+0000","last_clean":"2024-09-17T04:33:20.930502+0000","last_became_active":"2024-09-17T04:33:18.687839+0000","last_became_peered":"2024-09-17T04:33:18.687839+0000","last_unstale":"2024-09-17T04:33:20.930502+0000","last_undegraded":"2024-09-17T04:33:20.930502+0000","last_fullsized":"2024-09-17T04:33:20.930502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:27.606246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690522+0000","last_change":"2024-09-17T04:33:19.695246+0000","last_active":"2024-09-17T04:33:20.690522+0000","last_peered":"2024-09-17T04:33:20.690522+0000","last_clean":"2024-09-17T04:33:20.690522+0000","last_became_active":"2024-09-17T04:33:19.694929+0000","last_became_peered":"2024-09-17T04:33:19.694929+0000","last_unstale":"2024-09-17T04:33:20.690522+0000","last_undegraded":"2024-09-17T04:33:20.690522+0000","last_fullsized":"2024-09-17T04:33:20.690522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:38:26.907899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674093+0000","last_change":"2024-09-17T04:33:18.682334+0000","last_active":"2024-09-17T04:33:21.674093+0000","last_peered":"2024-09-17T04:33:21.674093+0000","last_clean":"2024-09-17T04:33:21.674093+0000","last_became_active":"2024-09-17T04:33:18.681988+0000","last_became_peered":"2024-09-17T04:33:18.681988+0000","last_unstale":"2024-09-17T04:33:21.674093+0000","last_undegraded":"2024-09-17T04:33:21.674093+0000","last_fullsized":"2024-09-17T04:33:21.674093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:40.963166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930502+0000","last_change":"2024-09-17T04:33:19.694190+0000","last_active":"2024-09-17T04:33:20.930502+0000","last_peered":"2024-09-17T04:33:20.930502+0000","last_clean":"2024-09-17T04:33:20.930502+0000","last_became_active":"2024-09-17T04:33:19.691073+0000","last_became_peered":"2024-09-17T04:33:19.691073+0000","last_unstale":"2024-09-17T04:33:20.930502+0000","last_undegraded":"2024-09-17T04:33:20.930502+0000","last_fullsized":"2024-09-17T04:33:20.930502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:04:44.157056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690492+0000","last_change":"2024-09-17T04:33:18.687984+0000","last_active":"2024-09-17T04:33:20.690492+0000","last_peered":"2024-09-17T04:33:20.690492+0000","last_clean":"2024-09-17T04:33:20.690492+0000","last_became_active":"2024-09-17T04:33:18.687519+0000","last_became_peered":"2024-09-17T04:33:18.687519+0000","last_unstale":"2024-09-17T04:33:20.690492+0000","last_undegraded":"2024-09-17T04:33:20.690492+0000","last_fullsized":"2024-09-17T04:33:20.690492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:45.809989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674114+0000","last_change":"2024-09-17T04:33:19.688598+0000","last_active":"2024-09-17T04:33:21.674114+0000","last_peered":"2024-09-17T04:33:21.674114+0000","last_clean":"2024-09-17T04:33:21.674114+0000","last_became_active":"2024-09-17T04:33:19.688240+0000","last_became_peered":"2024-09-17T04:33:19.688240+0000","last_unstale":"2024-09-17T04:33:21.674114+0000","last_undegraded":"2024-09-17T04:33:21.674114+0000","last_fullsized":"2024-09-17T04:33:21.674114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:39.340136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.676358+0000","last_change":"2024-09-17T04:33:18.693000+0000","last_active":"2024-09-17T04:33:21.676358+0000","last_peered":"2024-09-17T04:33:21.676358+0000","last_clean":"2024-09-17T04:33:21.676358+0000","last_became_active":"2024-09-17T04:33:18.692677+0000","last_became_peered":"2024-09-17T04:33:18.692677+0000","last_unstale":"2024-09-17T04:33:21.676358+0000","last_undegraded":"2024-09-17T04:33:21.676358+0000","last_fullsized":"2024-09-17T04:33:21.676358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:45:37.943381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035886+0000","last_change":"2024-09-17T04:33:19.695017+0000","last_active":"2024-09-17T04:33:22.035886+0000","last_peered":"2024-09-17T04:33:22.035886+0000","last_clean":"2024-09-17T04:33:22.035886+0000","last_became_active":"2024-09-17T04:33:19.694621+0000","last_became_peered":"2024-09-17T04:33:19.694621+0000","last_unstale":"2024-09-17T04:33:22.035886+0000","last_undegraded":"2024-09-17T04:33:22.035886+0000","last_fullsized":"2024-09-17T04:33:22.035886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:38.326621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677397+0000","last_change":"2024-09-17T04:33:18.685926+0000","last_active":"2024-09-17T04:33:21.677397+0000","last_peered":"2024-09-17T04:33:21.677397+0000","last_clean":"2024-09-17T04:33:21.677397+0000","last_became_active":"2024-09-17T04:33:18.685598+0000","last_became_peered":"2024-09-17T04:33:18.685598+0000","last_unstale":"2024-09-17T04:33:21.677397+0000","last_undegraded":"2024-09-17T04:33:21.677397+0000","last_fullsized":"2024-09-17T04:33:21.677397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:45:37.943381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035362+0000","last_change":"2024-09-17T04:33:19.681836+0000","last_active":"2024-09-17T04:33:22.035362+0000","last_peered":"2024-09-17T04:33:22.035362+0000","last_clean":"2024-09-17T04:33:22.035362+0000","last_became_active":"2024-09-17T04:33:19.681085+0000","last_became_peered":"2024-09-17T04:33:19.681085+0000","last_unstale":"2024-09-17T04:33:22.035362+0000","last_undegraded":"2024-09-17T04:33:22.035362+0000","last_fullsized":"2024-09-17T04:33:22.035362+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:58.841388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.689553+0000","last_change":"2024-09-17T04:33:18.688337+0000","last_active":"2024-09-17T04:33:20.689553+0000","last_peered":"2024-09-17T04:33:20.689553+0000","last_clean":"2024-09-17T04:33:20.689553+0000","last_became_active":"2024-09-17T04:33:18.687331+0000","last_became_peered":"2024-09-17T04:33:18.687331+0000","last_unstale":"2024-09-17T04:33:20.689553+0000","last_undegraded":"2024-09-17T04:33:20.689553+0000","last_fullsized":"2024-09-17T04:33:20.689553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:27.606246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691952+0000","last_change":"2024-09-17T04:33:19.690673+0000","last_active":"2024-09-17T04:33:20.691952+0000","last_peered":"2024-09-17T04:33:20.691952+0000","last_clean":"2024-09-17T04:33:20.691952+0000","last_became_active":"2024-09-17T04:33:19.690356+0000","last_became_peered":"2024-09-17T04:33:19.690356+0000","last_unstale":"2024-09-17T04:33:20.691952+0000","last_undegraded":"2024-09-17T04:33:20.691952+0000","last_fullsized":"2024-09-17T04:33:20.691952+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:42.127433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674278+0000","last_change":"2024-09-17T04:33:18.686258+0000","last_active":"2024-09-17T04:33:21.674278+0000","last_peered":"2024-09-17T04:33:21.674278+0000","last_clean":"2024-09-17T04:33:21.674278+0000","last_became_active":"2024-09-17T04:33:18.685943+0000","last_became_peered":"2024-09-17T04:33:18.685943+0000","last_unstale":"2024-09-17T04:33:21.674278+0000","last_undegraded":"2024-09-17T04:33:21.674278+0000","last_fullsized":"2024-09-17T04:33:21.674278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:23.445663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035293+0000","last_change":"2024-09-17T04:33:19.684510+0000","last_active":"2024-09-17T04:33:22.035293+0000","last_peered":"2024-09-17T04:33:22.035293+0000","last_clean":"2024-09-17T04:33:22.035293+0000","last_became_active":"2024-09-17T04:33:19.683614+0000","last_became_peered":"2024-09-17T04:33:19.683614+0000","last_unstale":"2024-09-17T04:33:22.035293+0000","last_undegraded":"2024-09-17T04:33:22.035293+0000","last_fullsized":"2024-09-17T04:33:22.035293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:22.696900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674344+0000","last_change":"2024-09-17T04:33:18.684287+0000","last_active":"2024-09-17T04:33:21.674344+0000","last_peered":"2024-09-17T04:33:21.674344+0000","last_clean":"2024-09-17T04:33:21.674344+0000","last_became_active":"2024-09-17T04:33:18.683822+0000","last_became_peered":"2024-09-17T04:33:18.683822+0000","last_unstale":"2024-09-17T04:33:21.674344+0000","last_undegraded":"2024-09-17T04:33:21.674344+0000","last_fullsized":"2024-09-17T04:33:21.674344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:53:28.012165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036101+0000","last_change":"2024-09-17T04:33:19.699298+0000","last_active":"2024-09-17T04:33:22.036101+0000","last_peered":"2024-09-17T04:33:22.036101+0000","last_clean":"2024-09-17T04:33:22.036101+0000","last_became_active":"2024-09-17T04:33:19.698925+0000","last_became_peered":"2024-09-17T04:33:19.698925+0000","last_unstale":"2024-09-17T04:33:22.036101+0000","last_undegraded":"2024-09-17T04:33:22.036101+0000","last_fullsized":"2024-09-17T04:33:22.036101+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:13.459991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690689+0000","last_change":"2024-09-17T04:33:18.685941+0000","last_active":"2024-09-17T04:33:20.690689+0000","last_peered":"2024-09-17T04:33:20.690689+0000","last_clean":"2024-09-17T04:33:20.690689+0000","last_became_active":"2024-09-17T04:33:18.685484+0000","last_became_peered":"2024-09-17T04:33:18.685484+0000","last_unstale":"2024-09-17T04:33:20.690689+0000","last_undegraded":"2024-09-17T04:33:20.690689+0000","last_fullsized":"2024-09-17T04:33:20.690689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:40.963166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677177+0000","last_change":"2024-09-17T04:33:19.685032+0000","last_active":"2024-09-17T04:33:21.677177+0000","last_peered":"2024-09-17T04:33:21.677177+0000","last_clean":"2024-09-17T04:33:21.677177+0000","last_became_active":"2024-09-17T04:33:19.683930+0000","last_became_peered":"2024-09-17T04:33:19.683930+0000","last_unstale":"2024-09-17T04:33:21.677177+0000","last_undegraded":"2024-09-17T04:33:21.677177+0000","last_fullsized":"2024-09-17T04:33:21.677177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:16.814580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930467+0000","last_change":"2024-09-17T04:33:19.691762+0000","last_active":"2024-09-17T04:33:20.930467+0000","last_peered":"2024-09-17T04:33:20.930467+0000","last_clean":"2024-09-17T04:33:20.930467+0000","last_became_active":"2024-09-17T04:33:19.690765+0000","last_became_peered":"2024-09-17T04:33:19.690765+0000","last_unstale":"2024-09-17T04:33:20.930467+0000","last_undegraded":"2024-09-17T04:33:20.930467+0000","last_fullsized":"2024-09-17T04:33:20.930467+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:59.693472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.689803+0000","last_change":"2024-09-17T04:33:18.694221+0000","last_active":"2024-09-17T04:33:20.689803+0000","last_peered":"2024-09-17T04:33:20.689803+0000","last_clean":"2024-09-17T04:33:20.689803+0000","last_became_active":"2024-09-17T04:33:18.693906+0000","last_became_peered":"2024-09-17T04:33:18.693906+0000","last_unstale":"2024-09-17T04:33:20.689803+0000","last_undegraded":"2024-09-17T04:33:20.689803+0000","last_fullsized":"2024-09-17T04:33:20.689803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:53.979348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930455+0000","last_change":"2024-09-17T04:33:19.692952+0000","last_active":"2024-09-17T04:33:20.930455+0000","last_peered":"2024-09-17T04:33:20.930455+0000","last_clean":"2024-09-17T04:33:20.930455+0000","last_became_active":"2024-09-17T04:33:19.692612+0000","last_became_peered":"2024-09-17T04:33:19.692612+0000","last_unstale":"2024-09-17T04:33:20.930455+0000","last_undegraded":"2024-09-17T04:33:20.930455+0000","last_fullsized":"2024-09-17T04:33:20.930455+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:12:48.590039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690377+0000","last_change":"2024-09-17T04:33:18.692471+0000","last_active":"2024-09-17T04:33:20.690377+0000","last_peered":"2024-09-17T04:33:20.690377+0000","last_clean":"2024-09-17T04:33:20.690377+0000","last_became_active":"2024-09-17T04:33:18.692129+0000","last_became_peered":"2024-09-17T04:33:18.692129+0000","last_unstale":"2024-09-17T04:33:20.690377+0000","last_undegraded":"2024-09-17T04:33:20.690377+0000","last_fullsized":"2024-09-17T04:33:20.690377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:14:47.283001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035813+0000","last_change":"2024-09-17T04:33:19.691269+0000","last_active":"2024-09-17T04:33:22.035813+0000","last_peered":"2024-09-17T04:33:22.035813+0000","last_clean":"2024-09-17T04:33:22.035813+0000","last_became_active":"2024-09-17T04:33:19.690548+0000","last_became_peered":"2024-09-17T04:33:19.690548+0000","last_unstale":"2024-09-17T04:33:22.035813+0000","last_undegraded":"2024-09-17T04:33:22.035813+0000","last_fullsized":"2024-09-17T04:33:22.035813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:03.560173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036319+0000","last_change":"2024-09-17T04:33:18.689670+0000","last_active":"2024-09-17T04:33:22.036319+0000","last_peered":"2024-09-17T04:33:22.036319+0000","last_clean":"2024-09-17T04:33:22.036319+0000","last_became_active":"2024-09-17T04:33:18.689277+0000","last_became_peered":"2024-09-17T04:33:18.689277+0000","last_unstale":"2024-09-17T04:33:22.036319+0000","last_undegraded":"2024-09-17T04:33:22.036319+0000","last_fullsized":"2024-09-17T04:33:22.036319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:30.774768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036234+0000","last_change":"2024-09-17T04:33:19.686603+0000","last_active":"2024-09-17T04:33:22.036234+0000","last_peered":"2024-09-17T04:33:22.036234+0000","last_clean":"2024-09-17T04:33:22.036234+0000","last_became_active":"2024-09-17T04:33:19.686252+0000","last_became_peered":"2024-09-17T04:33:19.686252+0000","last_unstale":"2024-09-17T04:33:22.036234+0000","last_undegraded":"2024-09-17T04:33:22.036234+0000","last_fullsized":"2024-09-17T04:33:22.036234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:44.199782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035982+0000","last_change":"2024-09-17T04:33:18.683562+0000","last_active":"2024-09-17T04:33:22.035982+0000","last_peered":"2024-09-17T04:33:22.035982+0000","last_clean":"2024-09-17T04:33:22.035982+0000","last_became_active":"2024-09-17T04:33:18.683155+0000","last_became_peered":"2024-09-17T04:33:18.683155+0000","last_unstale":"2024-09-17T04:33:22.035982+0000","last_undegraded":"2024-09-17T04:33:22.035982+0000","last_fullsized":"2024-09-17T04:33:22.035982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:59:56.848238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":823296,"data_stored":669858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":80520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":78998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":237568,"data_stored":79034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":74220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":78908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":79068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":669914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:33:24.119 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T04:33:24.119 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-17T04:33:24.120 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-17T04:33:24.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.273+0000 7f76fea4a640 1 -- 172.21.15.37:0/139332935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76e00042f0 msgr2=0x7f76e0004820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:24.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.273+0000 7f76fea4a640 1 --2- 172.21.15.37:0/139332935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76e00042f0 0x7f76e0004820 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f76ec006f50 tx=0x7f76e0005830 comp rx=0 tx=0).stop 2024-09-17T04:33:24.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.273+0000 7f76fcfe7640 1 -- 172.21.15.37:0/139332935 reap_dead start 2024-09-17T04:33:24.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.273+0000 7f76fea4a640 1 -- 172.21.15.37:0/139332935 shutdown_connections 2024-09-17T04:33:24.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.273+0000 7f76fea4a640 1 -- 172.21.15.37:0/139332935 >> 172.21.15.37:0/139332935 conn(0x7f76f80d04e0 msgr2=0x7f76f80d08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:24.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.273+0000 7f76fea4a640 1 -- 172.21.15.37:0/139332935 shutdown_connections 2024-09-17T04:33:24.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.273+0000 7f76fea4a640 1 -- 172.21.15.37:0/139332935 wait complete. 2024-09-17T04:33:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.275+0000 7f76fea4a640 1 Processor -- start 2024-09-17T04:33:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.275+0000 7f76fea4a640 1 -- start start 2024-09-17T04:33:24.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.276+0000 7f76fea4a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76e00042f0 0x7f76f80af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:24.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.276+0000 7f76fea4a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76f81f2460 0x7f76f81dfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:24.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.276+0000 7f76fea4a640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f76f81e01e0 0x7f76f81ddda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:24.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.276+0000 7f76fea4a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f76f80d01c0 con 0x7f76e00042f0 2024-09-17T04:33:24.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.276+0000 7f76fea4a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f76f80d1330 con 0x7f76f81f2460 2024-09-17T04:33:24.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.276+0000 7f76fea4a640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f76f80d0340 con 0x7f76f81e01e0 2024-09-17T04:33:24.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.276+0000 7f76f7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76e00042f0 0x7f76f80af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:24.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.276+0000 7f76f7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76e00042f0 0x7f76f80af6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40774/0 (socket says 172.21.15.37:40774) 2024-09-17T04:33:24.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.276+0000 7f76f7fff640 1 -- 172.21.15.37:0/3574658584 learned_addr learned my addr 172.21.15.37:0/3574658584 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:24.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.277+0000 7f76f77fe640 1 --2- 172.21.15.37:0/3574658584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76f81f2460 0x7f76f81dfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:24.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.277+0000 7f76fcfe7640 1 --2- 172.21.15.37:0/3574658584 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f76f81e01e0 0x7f76f81ddda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:24.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.277+0000 7f76f7fff640 1 -- 172.21.15.37:0/3574658584 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f76f81e01e0 msgr2=0x7f76f81ddda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:24.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.277+0000 7f76f7fff640 1 --2- 172.21.15.37:0/3574658584 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f76f81e01e0 0x7f76f81ddda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:24.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.277+0000 7f76f7fff640 1 -- 172.21.15.37:0/3574658584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76f81f2460 msgr2=0x7f76f81dfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:24.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.277+0000 7f76f7fff640 1 --2- 172.21.15.37:0/3574658584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76f81f2460 0x7f76f81dfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:24.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.277+0000 7f76f7fff640 1 -- 172.21.15.37:0/3574658584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f81f2b90 con 0x7f76e00042f0 2024-09-17T04:33:24.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.277+0000 7f76f7fff640 1 --2- 172.21.15.37:0/3574658584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76e00042f0 0x7f76f80af6a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f76e8011bc0 tx=0x7f76e800e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:24.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.277+0000 7f76f4ff9640 1 -- 172.21.15.37:0/3574658584 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8004030 con 0x7f76e00042f0 2024-09-17T04:33:24.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.277+0000 7f76f4ff9640 1 -- 172.21.15.37:0/3574658584 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76e80041d0 con 0x7f76e00042f0 2024-09-17T04:33:24.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.277+0000 7f76fea4a640 1 -- 172.21.15.37:0/3574658584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f81f2e70 con 0x7f76e00042f0 2024-09-17T04:33:24.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.277+0000 7f76f4ff9640 1 -- 172.21.15.37:0/3574658584 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e80035f0 con 0x7f76e00042f0 2024-09-17T04:33:24.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.277+0000 7f76fea4a640 1 -- 172.21.15.37:0/3574658584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f81f39d0 con 0x7f76e00042f0 2024-09-17T04:33:24.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.278+0000 7f76fea4a640 1 -- 172.21.15.37:0/3574658584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76f80b1230 con 0x7f76e00042f0 2024-09-17T04:33:24.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.279+0000 7f76f4ff9640 1 -- 172.21.15.37:0/3574658584 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f76e8003790 con 0x7f76e00042f0 2024-09-17T04:33:24.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.279+0000 7f76f4ff9640 1 --2- 172.21.15.37:0/3574658584 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f76e4078bc0 0x7f76e407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:24.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.279+0000 7f76f4ff9640 1 -- 172.21.15.37:0/3574658584 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f76e809d8e0 con 0x7f76e00042f0 2024-09-17T04:33:24.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.280+0000 7f76f77fe640 1 --2- 172.21.15.37:0/3574658584 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f76e4078bc0 0x7f76e407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:24.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.280+0000 7f76f77fe640 1 --2- 172.21.15.37:0/3574658584 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f76e4078bc0 0x7f76e407b0d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f76ec033000 tx=0x7f76ec02d000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:24.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.284+0000 7f76f4ff9640 1 -- 172.21.15.37:0/3574658584 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f76e8065d20 con 0x7f76e00042f0 2024-09-17T04:33:24.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.473+0000 7f76fea4a640 1 -- 172.21.15.37:0/3574658584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f76f80b6010 con 0x7f76e00042f0 2024-09-17T04:33:24.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.473+0000 7f76f4ff9640 1 -- 172.21.15.37:0/3574658584 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v31) ==== 86+0+160 (secure 0 0 0) 0x7f76e8065620 con 0x7f76e00042f0 2024-09-17T04:33:24.474 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:24.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.476+0000 7f76fea4a640 1 -- 172.21.15.37:0/3574658584 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f76e4078bc0 msgr2=0x7f76e407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:24.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.476+0000 7f76fea4a640 1 --2- 172.21.15.37:0/3574658584 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f76e4078bc0 0x7f76e407b0d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f76ec033000 tx=0x7f76ec02d000 comp rx=0 tx=0).stop 2024-09-17T04:33:24.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.476+0000 7f76fea4a640 1 -- 172.21.15.37:0/3574658584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76e00042f0 msgr2=0x7f76f80af6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:24.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.476+0000 7f76fea4a640 1 --2- 172.21.15.37:0/3574658584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76e00042f0 0x7f76f80af6a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f76e8011bc0 tx=0x7f76e800e780 comp rx=0 tx=0).stop 2024-09-17T04:33:24.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.476+0000 7f76fea4a640 1 -- 172.21.15.37:0/3574658584 shutdown_connections 2024-09-17T04:33:24.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.476+0000 7f76fea4a640 1 --2- 172.21.15.37:0/3574658584 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f76e4078bc0 0x7f76e407b0d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:24.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.476+0000 7f76fea4a640 1 --2- 172.21.15.37:0/3574658584 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f76f81e01e0 0x7f76f81ddda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:24.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.476+0000 7f76fea4a640 1 --2- 172.21.15.37:0/3574658584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76f81f2460 0x7f76f81dfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:24.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.476+0000 7f76fea4a640 1 --2- 172.21.15.37:0/3574658584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76e00042f0 0x7f76f80af6a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:24.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.476+0000 7f76fea4a640 1 -- 172.21.15.37:0/3574658584 >> 172.21.15.37:0/3574658584 conn(0x7f76f80d04e0 msgr2=0x7f76f80d08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:24.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.476+0000 7f76fea4a640 1 -- 172.21.15.37:0/3574658584 shutdown_connections 2024-09-17T04:33:24.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.476+0000 7f76fea4a640 1 -- 172.21.15.37:0/3574658584 wait complete. 2024-09-17T04:33:24.487 INFO:teuthology.orchestra.run.smithi037.stdout:[{"version":31,"timestamp":"2024-09-17T04:33:06.456414+0000","name":"","changes":[{"name":"osd.0/osd_mclock_max_capacity_iops_ssd","new_value":"3939.615108"}]}] 2024-09-17T04:33:24.487 INFO:tasks.ceph_manager:config epoch is 31 2024-09-17T04:33:24.488 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-17T04:33:24.488 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-17T04:33:24.488 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-17T04:33:24.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.604+0000 7fb2ad398640 1 -- 172.21.15.37:0/1619630804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2a80b6650 msgr2=0x7fb2a815e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:24.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.604+0000 7fb2ad398640 1 --2- 172.21.15.37:0/1619630804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2a80b6650 0x7fb2a815e8e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb29c005fd0 tx=0x7fb29c02a540 comp rx=0 tx=0).stop 2024-09-17T04:33:24.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.604+0000 7fb2ad398640 1 -- 172.21.15.37:0/1619630804 shutdown_connections 2024-09-17T04:33:24.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.604+0000 7fb2ad398640 1 --2- 172.21.15.37:0/1619630804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2a80b6650 0x7fb2a815e8e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:24.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.604+0000 7fb2ad398640 1 -- 172.21.15.37:0/1619630804 >> 172.21.15.37:0/1619630804 conn(0x7fb2a80d44c0 msgr2=0x7fb2a80d48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:24.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.604+0000 7fb2ad398640 1 -- 172.21.15.37:0/1619630804 shutdown_connections 2024-09-17T04:33:24.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.604+0000 7fb2ad398640 1 -- 172.21.15.37:0/1619630804 wait complete. 2024-09-17T04:33:24.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.606+0000 7fb2ad398640 1 Processor -- start 2024-09-17T04:33:24.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.606+0000 7fb2ad398640 1 -- start start 2024-09-17T04:33:24.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.607+0000 7fb2ad398640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb2a80cef40 0x7fb2a81f5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:24.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.607+0000 7fb2ad398640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2a81f8550 0x7fb2a81e8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:24.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.607+0000 7fb2ad398640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2a81f8960 0x7fb2a81e70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:24.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.607+0000 7fb2ad398640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a80b8400 con 0x7fb2a81f8550 2024-09-17T04:33:24.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.607+0000 7fb2ad398640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a80b8280 con 0x7fb2a81f8960 2024-09-17T04:33:24.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.607+0000 7fb2ad398640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fb2a80b8580 con 0x7fb2a80cef40 2024-09-17T04:33:24.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.607+0000 7fb2a67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2a81f8550 0x7fb2a81e8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:24.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.607+0000 7fb2a67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2a81f8550 0x7fb2a81e8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40786/0 (socket says 172.21.15.37:40786) 2024-09-17T04:33:24.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.607+0000 7fb2a67fc640 1 -- 172.21.15.37:0/4112315661 learned_addr learned my addr 172.21.15.37:0/4112315661 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:24.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.607+0000 7fb2a77fe640 1 --2- 172.21.15.37:0/4112315661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2a81f8960 0x7fb2a81e70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:24.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.607+0000 7fb2a6ffd640 1 --2- 172.21.15.37:0/4112315661 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb2a80cef40 0x7fb2a81f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:24.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.607+0000 7fb2a67fc640 1 -- 172.21.15.37:0/4112315661 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb2a80cef40 msgr2=0x7fb2a81f5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:24.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.607+0000 7fb2a67fc640 1 --2- 172.21.15.37:0/4112315661 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb2a80cef40 0x7fb2a81f5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:24.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.607+0000 7fb2a67fc640 1 -- 172.21.15.37:0/4112315661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2a81f8960 msgr2=0x7fb2a81e70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:24.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.607+0000 7fb2a67fc640 1 --2- 172.21.15.37:0/4112315661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2a81f8960 0x7fb2a81e70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:24.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.607+0000 7fb2a67fc640 1 -- 172.21.15.37:0/4112315661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a81f8ff0 con 0x7fb2a81f8550 2024-09-17T04:33:24.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.607+0000 7fb2a77fe640 1 --2- 172.21.15.37:0/4112315661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2a81f8960 0x7fb2a81e70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:24.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.607+0000 7fb2a6ffd640 1 --2- 172.21.15.37:0/4112315661 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb2a80cef40 0x7fb2a81f5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:24.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.608+0000 7fb2a67fc640 1 --2- 172.21.15.37:0/4112315661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2a81f8550 0x7fb2a81e8270 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb29c034040 tx=0x7fb29c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:24.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.608+0000 7fb28f7fe640 1 -- 172.21.15.37:0/4112315661 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c03e030 con 0x7fb2a81f8550 2024-09-17T04:33:24.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.608+0000 7fb28f7fe640 1 -- 172.21.15.37:0/4112315661 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb29c0047a0 con 0x7fb2a81f8550 2024-09-17T04:33:24.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.608+0000 7fb2ad398640 1 -- 172.21.15.37:0/4112315661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a81f92d0 con 0x7fb2a81f8550 2024-09-17T04:33:24.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.608+0000 7fb28f7fe640 1 -- 172.21.15.37:0/4112315661 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c035040 con 0x7fb2a81f8550 2024-09-17T04:33:24.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.608+0000 7fb2ad398640 1 -- 172.21.15.37:0/4112315661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a81f5550 con 0x7fb2a81f8550 2024-09-17T04:33:24.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.609+0000 7fb2ad398640 1 -- 172.21.15.37:0/4112315661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2a80d3fe0 con 0x7fb2a81f8550 2024-09-17T04:33:24.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.609+0000 7fb28f7fe640 1 -- 172.21.15.37:0/4112315661 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb29c041030 con 0x7fb2a81f8550 2024-09-17T04:33:24.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.610+0000 7fb28f7fe640 1 --2- 172.21.15.37:0/4112315661 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb294078b90 0x7fb29407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:24.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.610+0000 7fb28f7fe640 1 -- 172.21.15.37:0/4112315661 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb29c0bb520 con 0x7fb2a81f8550 2024-09-17T04:33:24.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.610+0000 7fb2a6ffd640 1 --2- 172.21.15.37:0/4112315661 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb294078b90 0x7fb29407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:24.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.611+0000 7fb2a6ffd640 1 --2- 172.21.15.37:0/4112315661 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb294078b90 0x7fb29407b0a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb29800a830 tx=0x7fb298008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:24.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.615+0000 7fb28f7fe640 1 -- 172.21.15.37:0/4112315661 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb29c0839f0 con 0x7fb2a81f8550 2024-09-17T04:33:24.653 INFO:tasks.ceph.mds.b.smithi092.stderr:2024-09-17T04:33:24.653+0000 7fc8e3da1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T04:33:24.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.855+0000 7fb2ad398640 1 -- 172.21.15.37:0/4112315661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fb2a81f8d80 con 0x7fb2a81f8550 2024-09-17T04:33:24.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.858+0000 7fb28f7fe640 1 -- 172.21.15.37:0/4112315661 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+194711 (secure 0 0 0) 0x7fb29c0832f0 con 0x7fb2a81f8550 2024-09-17T04:33:24.860 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:24.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.861+0000 7fb2ad398640 1 -- 172.21.15.37:0/4112315661 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb294078b90 msgr2=0x7fb29407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:24.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.861+0000 7fb2ad398640 1 --2- 172.21.15.37:0/4112315661 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb294078b90 0x7fb29407b0a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb29800a830 tx=0x7fb298008040 comp rx=0 tx=0).stop 2024-09-17T04:33:24.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.861+0000 7fb2ad398640 1 -- 172.21.15.37:0/4112315661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2a81f8550 msgr2=0x7fb2a81e8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:24.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.861+0000 7fb2ad398640 1 --2- 172.21.15.37:0/4112315661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2a81f8550 0x7fb2a81e8270 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb29c034040 tx=0x7fb29c003970 comp rx=0 tx=0).stop 2024-09-17T04:33:24.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.861+0000 7fb2ad398640 1 -- 172.21.15.37:0/4112315661 shutdown_connections 2024-09-17T04:33:24.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.861+0000 7fb2ad398640 1 --2- 172.21.15.37:0/4112315661 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb294078b90 0x7fb29407b0a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:24.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.861+0000 7fb2ad398640 1 --2- 172.21.15.37:0/4112315661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2a81f8960 0x7fb2a81e70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:24.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.861+0000 7fb2ad398640 1 --2- 172.21.15.37:0/4112315661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2a81f8550 0x7fb2a81e8270 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:24.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.861+0000 7fb2ad398640 1 --2- 172.21.15.37:0/4112315661 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb2a80cef40 0x7fb2a81f5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:24.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.861+0000 7fb2ad398640 1 -- 172.21.15.37:0/4112315661 >> 172.21.15.37:0/4112315661 conn(0x7fb2a80d44c0 msgr2=0x7fb2a80d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:24.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.861+0000 7fb2ad398640 1 -- 172.21.15.37:0/4112315661 shutdown_connections 2024-09-17T04:33:24.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.861+0000 7fb2ad398640 1 -- 172.21.15.37:0/4112315661 wait complete. 2024-09-17T04:33:24.872 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":864337835},{"type":"v1","addr":"172.21.15.92:6801","nonce":864337835}]},"active_addr":"172.21.15.92:6801/864337835","active_change":"2024-09-17T04:32:58.434536+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4116,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":969483958}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":1819355916}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":3906198958}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":3158995183}]}]} 2024-09-17T04:33:24.876 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-17T04:33:24.877 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-17T04:33:24.877 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:33:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.993+0000 7ff26599b640 1 -- 172.21.15.37:0/655454205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2600b66d0 msgr2=0x7ff2601627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.993+0000 7ff26599b640 1 --2- 172.21.15.37:0/655454205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2600b66d0 0x7ff2601627b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff2600cb510 tx=0x7ff24802a530 comp rx=0 tx=0).stop 2024-09-17T04:33:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.994+0000 7ff26599b640 1 -- 172.21.15.37:0/655454205 shutdown_connections 2024-09-17T04:33:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.994+0000 7ff26599b640 1 --2- 172.21.15.37:0/655454205 >> v2:172.21.15.92:3300/0 conn(0x7ff260162cf0 0x7ff2600d2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.994+0000 7ff26599b640 1 --2- 172.21.15.37:0/655454205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2600b66d0 0x7ff2601627b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.994+0000 7ff26599b640 1 -- 172.21.15.37:0/655454205 >> 172.21.15.37:0/655454205 conn(0x7ff2600d4540 msgr2=0x7ff2600d4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.994+0000 7ff26599b640 1 -- 172.21.15.37:0/655454205 shutdown_connections 2024-09-17T04:33:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.994+0000 7ff26599b640 1 -- 172.21.15.37:0/655454205 wait complete. 2024-09-17T04:33:24.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.995+0000 7ff26599b640 1 Processor -- start 2024-09-17T04:33:24.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.995+0000 7ff26599b640 1 -- start start 2024-09-17T04:33:24.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.997+0000 7ff26599b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff26015e830 0x7ff26014a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:24.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.997+0000 7ff26599b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff260162cf0 0x7ff26014aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:24.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.997+0000 7ff26599b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff26015bc80 0x7ff26014b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:24.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.997+0000 7ff26599b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff2600d3030 con 0x7ff260162cf0 2024-09-17T04:33:24.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.997+0000 7ff26599b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff2600d2eb0 con 0x7ff26015bc80 2024-09-17T04:33:24.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.997+0000 7ff26599b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7ff2600d31b0 con 0x7ff26015e830 2024-09-17T04:33:24.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.997+0000 7ff257fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff260162cf0 0x7ff26014aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:24.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.997+0000 7ff257fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff260162cf0 0x7ff26014aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40818/0 (socket says 172.21.15.37:40818) 2024-09-17T04:33:24.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.997+0000 7ff257fff640 1 -- 172.21.15.37:0/519484282 learned_addr learned my addr 172.21.15.37:0/519484282 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:24.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.997+0000 7ff25f7fe640 1 --2- 172.21.15.37:0/519484282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff26015bc80 0x7ff26014b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:24.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.997+0000 7ff25effd640 1 --2- 172.21.15.37:0/519484282 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff26015e830 0x7ff26014a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:24.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.997+0000 7ff257fff640 1 -- 172.21.15.37:0/519484282 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff26015e830 msgr2=0x7ff26014a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:24.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.997+0000 7ff257fff640 1 --2- 172.21.15.37:0/519484282 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff26015e830 0x7ff26014a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:24.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.997+0000 7ff257fff640 1 -- 172.21.15.37:0/519484282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff26015bc80 msgr2=0x7ff26014b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:24.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.997+0000 7ff257fff640 1 --2- 172.21.15.37:0/519484282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff26015bc80 0x7ff26014b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:24.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.997+0000 7ff257fff640 1 -- 172.21.15.37:0/519484282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2600cd640 con 0x7ff260162cf0 2024-09-17T04:33:24.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.997+0000 7ff257fff640 1 --2- 172.21.15.37:0/519484282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff260162cf0 0x7ff26014aba0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff24802ab20 tx=0x7ff248033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:24.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.998+0000 7ff25cff9640 1 -- 172.21.15.37:0/519484282 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff248031d60 con 0x7ff260162cf0 2024-09-17T04:33:24.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.998+0000 7ff25cff9640 1 -- 172.21.15.37:0/519484282 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff248033e10 con 0x7ff260162cf0 2024-09-17T04:33:24.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.998+0000 7ff26599b640 1 -- 172.21.15.37:0/519484282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2600cd920 con 0x7ff260162cf0 2024-09-17T04:33:24.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.998+0000 7ff25cff9640 1 -- 172.21.15.37:0/519484282 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff248005260 con 0x7ff260162cf0 2024-09-17T04:33:25.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.998+0000 7ff26599b640 1 -- 172.21.15.37:0/519484282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2600cdee0 con 0x7ff260162cf0 2024-09-17T04:33:25.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.999+0000 7ff26599b640 1 -- 172.21.15.37:0/519484282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2600d3660 con 0x7ff260162cf0 2024-09-17T04:33:25.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:24.999+0000 7ff25cff9640 1 -- 172.21.15.37:0/519484282 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff248005400 con 0x7ff260162cf0 2024-09-17T04:33:25.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.000+0000 7ff25cff9640 1 --2- 172.21.15.37:0/519484282 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff240078b90 0x7ff24007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.000+0000 7ff25cff9640 1 -- 172.21.15.37:0/519484282 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff2480bbea0 con 0x7ff260162cf0 2024-09-17T04:33:25.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.000+0000 7ff25effd640 1 --2- 172.21.15.37:0/519484282 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff240078b90 0x7ff24007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.001+0000 7ff25effd640 1 --2- 172.21.15.37:0/519484282 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff240078b90 0x7ff24007b0a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff250000f60 tx=0x7ff250009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:25.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.004+0000 7ff25cff9640 1 -- 172.21.15.37:0/519484282 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff24803f030 con 0x7ff260162cf0 2024-09-17T04:33:25.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.192+0000 7ff26599b640 1 -- 172.21.15.37:0/519484282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff2601628c0 con 0x7ff260162cf0 2024-09-17T04:33:25.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.193+0000 7ff25cff9640 1 -- 172.21.15.37:0/519484282 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16605 (secure 0 0 0) 0x7ff248031770 con 0x7ff260162cf0 2024-09-17T04:33:25.193 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:25.193 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":22,"fsid":"a908a603-430c-4209-8294-0c3460b35f42","created":"2024-09-17T04:32:54.139221+0000","modified":"2024-09-17T04:33:20.677493+0000","last_up_change":"2024-09-17T04:33:06.922210+0000","last_in_change":"2024-09-17T04:32:57.754118+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:33:08.461907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:33:16.958785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:18.013768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ea4b8d1-0d22-4dd9-b913-9df585ef24ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6809","nonce":2283587593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6811","nonce":2283587593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6815","nonce":2283587593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6813","nonce":2283587593}]},"public_addr":"172.21.15.37:6809/2283587593","cluster_addr":"172.21.15.37:6811/2283587593","heartbeat_back_addr":"172.21.15.37:6815/2283587593","heartbeat_front_addr":"172.21.15.37:6813/2283587593","state":["exists","up"]},{"osd":1,"uuid":"ac745a33-e8ad-4083-9c18-71bf81982b76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6825","nonce":2657955935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6827","nonce":2657955935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6831","nonce":2657955935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6829","nonce":2657955935}]},"public_addr":"172.21.15.37:6825/2657955935","cluster_addr":"172.21.15.37:6827/2657955935","heartbeat_back_addr":"172.21.15.37:6831/2657955935","heartbeat_front_addr":"172.21.15.37:6829/2657955935","state":["exists","up"]},{"osd":2,"uuid":"3c63f3d8-132a-4735-b0bc-d64313a46503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6817","nonce":579720176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6819","nonce":579720176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6823","nonce":579720176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6821","nonce":579720176}]},"public_addr":"172.21.15.37:6817/579720176","cluster_addr":"172.21.15.37:6819/579720176","heartbeat_back_addr":"172.21.15.37:6823/579720176","heartbeat_front_addr":"172.21.15.37:6821/579720176","state":["exists","up"]},{"osd":3,"uuid":"4f8c6ca2-ea1d-4610-9127-52c4dd8bc8e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6801","nonce":809975826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6803","nonce":809975826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6807","nonce":809975826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6805","nonce":809975826}]},"public_addr":"172.21.15.37:6801/809975826","cluster_addr":"172.21.15.37:6803/809975826","heartbeat_back_addr":"172.21.15.37:6807/809975826","heartbeat_front_addr":"172.21.15.37:6805/809975826","state":["exists","up"]},{"osd":4,"uuid":"63d0c4ab-f115-47b4-9611-12a10153473a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6819","nonce":2509785752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6821","nonce":2509785752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6825","nonce":2509785752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6823","nonce":2509785752}]},"public_addr":"172.21.15.92:6819/2509785752","cluster_addr":"172.21.15.92:6821/2509785752","heartbeat_back_addr":"172.21.15.92:6825/2509785752","heartbeat_front_addr":"172.21.15.92:6823/2509785752","state":["exists","up"]},{"osd":5,"uuid":"4c56c04d-0c07-4f76-9d75-6335121310ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6827","nonce":248263832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6829","nonce":248263832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6833","nonce":248263832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6831","nonce":248263832}]},"public_addr":"172.21.15.92:6827/248263832","cluster_addr":"172.21.15.92:6829/248263832","heartbeat_back_addr":"172.21.15.92:6833/248263832","heartbeat_front_addr":"172.21.15.92:6831/248263832","state":["exists","up"]},{"osd":6,"uuid":"b30eae7c-096a-4d24-ac51-96b8b13ecaee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6811","nonce":4078365985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6813","nonce":4078365985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6817","nonce":4078365985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6815","nonce":4078365985}]},"public_addr":"172.21.15.92:6811/4078365985","cluster_addr":"172.21.15.92:6813/4078365985","heartbeat_back_addr":"172.21.15.92:6817/4078365985","heartbeat_front_addr":"172.21.15.92:6815/4078365985","state":["exists","up"]},{"osd":7,"uuid":"62f12540-d374-4a4c-9ff7-63b7304fe77f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6803","nonce":3444787223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6805","nonce":3444787223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6809","nonce":3444787223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6807","nonce":3444787223}]},"public_addr":"172.21.15.92:6803/3444787223","cluster_addr":"172.21.15.92:6805/3444787223","heartbeat_back_addr":"172.21.15.92:6809/3444787223","heartbeat_front_addr":"172.21.15.92:6807/3444787223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.006420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.111135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.103325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.072656+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.200989+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.060455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.774773+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.865778+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:33:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.195+0000 7ff26599b640 1 -- 172.21.15.37:0/519484282 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff240078b90 msgr2=0x7ff24007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.195+0000 7ff26599b640 1 --2- 172.21.15.37:0/519484282 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff240078b90 0x7ff24007b0a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff250000f60 tx=0x7ff250009040 comp rx=0 tx=0).stop 2024-09-17T04:33:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.195+0000 7ff26599b640 1 -- 172.21.15.37:0/519484282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff260162cf0 msgr2=0x7ff26014aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.195+0000 7ff26599b640 1 --2- 172.21.15.37:0/519484282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff260162cf0 0x7ff26014aba0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff24802ab20 tx=0x7ff248033c20 comp rx=0 tx=0).stop 2024-09-17T04:33:25.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.195+0000 7ff26599b640 1 -- 172.21.15.37:0/519484282 shutdown_connections 2024-09-17T04:33:25.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.195+0000 7ff26599b640 1 --2- 172.21.15.37:0/519484282 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff240078b90 0x7ff24007b0a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.195+0000 7ff26599b640 1 --2- 172.21.15.37:0/519484282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff26015bc80 0x7ff26014b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.195+0000 7ff26599b640 1 --2- 172.21.15.37:0/519484282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff260162cf0 0x7ff26014aba0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.195+0000 7ff26599b640 1 --2- 172.21.15.37:0/519484282 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff26015e830 0x7ff26014a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.195+0000 7ff26599b640 1 -- 172.21.15.37:0/519484282 >> 172.21.15.37:0/519484282 conn(0x7ff2600d4540 msgr2=0x7ff2600d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:25.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.195+0000 7ff26599b640 1 -- 172.21.15.37:0/519484282 shutdown_connections 2024-09-17T04:33:25.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.195+0000 7ff26599b640 1 -- 172.21.15.37:0/519484282 wait complete. 2024-09-17T04:33:25.207 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-17T04:33:25.207 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:33:25.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.324+0000 7f914dc5c640 1 -- 172.21.15.37:0/615032837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91480b6650 msgr2=0x7f914815e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.324+0000 7f914dc5c640 1 --2- 172.21.15.37:0/615032837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91480b6650 0x7f914815e8e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f91480b38c0 tx=0x7f913002a560 comp rx=0 tx=0).stop 2024-09-17T04:33:25.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.324+0000 7f914dc5c640 1 -- 172.21.15.37:0/615032837 shutdown_connections 2024-09-17T04:33:25.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.324+0000 7f914dc5c640 1 --2- 172.21.15.37:0/615032837 >> v2:172.21.15.92:3300/0 conn(0x7f91480cef40 0x7f91480b7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.324+0000 7f914dc5c640 1 --2- 172.21.15.37:0/615032837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91480b6650 0x7f914815e8e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.324+0000 7f914dc5c640 1 -- 172.21.15.37:0/615032837 >> 172.21.15.37:0/615032837 conn(0x7f91480d44c0 msgr2=0x7f91480d48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:25.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.324+0000 7f914dc5c640 1 -- 172.21.15.37:0/615032837 shutdown_connections 2024-09-17T04:33:25.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.324+0000 7f914dc5c640 1 -- 172.21.15.37:0/615032837 wait complete. 2024-09-17T04:33:25.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.325+0000 7f914dc5c640 1 Processor -- start 2024-09-17T04:33:25.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.326+0000 7f914dc5c640 1 -- start start 2024-09-17T04:33:25.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.327+0000 7f914dc5c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91480cef40 0x7f91481f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.327+0000 7f914dc5c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f91480b2c00 0x7f9148160160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.327+0000 7f914dc5c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f914815c820 0x7f91480cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.327+0000 7f914dc5c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f91480b8230 con 0x7f91480cef40 2024-09-17T04:33:25.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.327+0000 7f914dc5c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f91480b80b0 con 0x7f914815c820 2024-09-17T04:33:25.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.327+0000 7f914dc5c640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f91480b83b0 con 0x7f91480b2c00 2024-09-17T04:33:25.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.327+0000 7f91477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91480cef40 0x7f91481f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.327+0000 7f91477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91480cef40 0x7f91481f4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45592/0 (socket says 172.21.15.37:45592) 2024-09-17T04:33:25.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.327+0000 7f91477fe640 1 -- 172.21.15.37:0/2590520334 learned_addr learned my addr 172.21.15.37:0/2590520334 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:25.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.327+0000 7f9147fff640 1 --2- 172.21.15.37:0/2590520334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f914815c820 0x7f91480cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.327+0000 7f913ffff640 1 --2- 172.21.15.37:0/2590520334 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f91480b2c00 0x7f9148160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.327+0000 7f91477fe640 1 -- 172.21.15.37:0/2590520334 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f91480b2c00 msgr2=0x7f9148160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.327+0000 7f91477fe640 1 --2- 172.21.15.37:0/2590520334 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f91480b2c00 0x7f9148160160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.327+0000 7f91477fe640 1 -- 172.21.15.37:0/2590520334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f914815c820 msgr2=0x7f91480cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.327+0000 7f91477fe640 1 --2- 172.21.15.37:0/2590520334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f914815c820 0x7f91480cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.327+0000 7f91477fe640 1 -- 172.21.15.37:0/2590520334 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91481f5c60 con 0x7f91480cef40 2024-09-17T04:33:25.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.327+0000 7f91477fe640 1 --2- 172.21.15.37:0/2590520334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91480cef40 0x7f91481f4c90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f913800c3d0 tx=0x7f913800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:25.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.328+0000 7f9144ff9640 1 -- 172.21.15.37:0/2590520334 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9138017070 con 0x7f91480cef40 2024-09-17T04:33:25.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.328+0000 7f9144ff9640 1 -- 172.21.15.37:0/2590520334 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91380053c0 con 0x7f91480cef40 2024-09-17T04:33:25.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.328+0000 7f9144ff9640 1 -- 172.21.15.37:0/2590520334 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9138003850 con 0x7f91480cef40 2024-09-17T04:33:25.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.328+0000 7f914dc5c640 1 -- 172.21.15.37:0/2590520334 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91481606a0 con 0x7f91480cef40 2024-09-17T04:33:25.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.328+0000 7f914dc5c640 1 -- 172.21.15.37:0/2590520334 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91481f56a0 con 0x7f91480cef40 2024-09-17T04:33:25.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.329+0000 7f914dc5c640 1 -- 172.21.15.37:0/2590520334 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9148160870 con 0x7f91480cef40 2024-09-17T04:33:25.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.329+0000 7f9144ff9640 1 -- 172.21.15.37:0/2590520334 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f913801a070 con 0x7f91480cef40 2024-09-17T04:33:25.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.330+0000 7f9144ff9640 1 --2- 172.21.15.37:0/2590520334 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9128078bc0 0x7f912807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.330+0000 7f9144ff9640 1 -- 172.21.15.37:0/2590520334 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f913809a000 con 0x7f91480cef40 2024-09-17T04:33:25.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.330+0000 7f913ffff640 1 --2- 172.21.15.37:0/2590520334 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9128078bc0 0x7f912807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.331+0000 7f913ffff640 1 --2- 172.21.15.37:0/2590520334 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9128078bc0 0x7f912807b0d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9130033000 tx=0x7f9130004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:25.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.334+0000 7f9144ff9640 1 -- 172.21.15.37:0/2590520334 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9138061df0 con 0x7f91480cef40 2024-09-17T04:33:25.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.524+0000 7f914dc5c640 1 -- 172.21.15.37:0/2590520334 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91480b8860 con 0x7f91480cef40 2024-09-17T04:33:25.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.525+0000 7f9144ff9640 1 -- 172.21.15.37:0/2590520334 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16605 (secure 0 0 0) 0x7f9138066820 con 0x7f91480cef40 2024-09-17T04:33:25.525 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:25.525 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":22,"fsid":"a908a603-430c-4209-8294-0c3460b35f42","created":"2024-09-17T04:32:54.139221+0000","modified":"2024-09-17T04:33:20.677493+0000","last_up_change":"2024-09-17T04:33:06.922210+0000","last_in_change":"2024-09-17T04:32:57.754118+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:33:08.461907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:33:16.958785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:18.013768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ea4b8d1-0d22-4dd9-b913-9df585ef24ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6809","nonce":2283587593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6811","nonce":2283587593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6815","nonce":2283587593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6813","nonce":2283587593}]},"public_addr":"172.21.15.37:6809/2283587593","cluster_addr":"172.21.15.37:6811/2283587593","heartbeat_back_addr":"172.21.15.37:6815/2283587593","heartbeat_front_addr":"172.21.15.37:6813/2283587593","state":["exists","up"]},{"osd":1,"uuid":"ac745a33-e8ad-4083-9c18-71bf81982b76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6825","nonce":2657955935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6827","nonce":2657955935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6831","nonce":2657955935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6829","nonce":2657955935}]},"public_addr":"172.21.15.37:6825/2657955935","cluster_addr":"172.21.15.37:6827/2657955935","heartbeat_back_addr":"172.21.15.37:6831/2657955935","heartbeat_front_addr":"172.21.15.37:6829/2657955935","state":["exists","up"]},{"osd":2,"uuid":"3c63f3d8-132a-4735-b0bc-d64313a46503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6817","nonce":579720176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6819","nonce":579720176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6823","nonce":579720176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6821","nonce":579720176}]},"public_addr":"172.21.15.37:6817/579720176","cluster_addr":"172.21.15.37:6819/579720176","heartbeat_back_addr":"172.21.15.37:6823/579720176","heartbeat_front_addr":"172.21.15.37:6821/579720176","state":["exists","up"]},{"osd":3,"uuid":"4f8c6ca2-ea1d-4610-9127-52c4dd8bc8e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6801","nonce":809975826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6803","nonce":809975826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6807","nonce":809975826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6805","nonce":809975826}]},"public_addr":"172.21.15.37:6801/809975826","cluster_addr":"172.21.15.37:6803/809975826","heartbeat_back_addr":"172.21.15.37:6807/809975826","heartbeat_front_addr":"172.21.15.37:6805/809975826","state":["exists","up"]},{"osd":4,"uuid":"63d0c4ab-f115-47b4-9611-12a10153473a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6819","nonce":2509785752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6821","nonce":2509785752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6825","nonce":2509785752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6823","nonce":2509785752}]},"public_addr":"172.21.15.92:6819/2509785752","cluster_addr":"172.21.15.92:6821/2509785752","heartbeat_back_addr":"172.21.15.92:6825/2509785752","heartbeat_front_addr":"172.21.15.92:6823/2509785752","state":["exists","up"]},{"osd":5,"uuid":"4c56c04d-0c07-4f76-9d75-6335121310ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6827","nonce":248263832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6829","nonce":248263832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6833","nonce":248263832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6831","nonce":248263832}]},"public_addr":"172.21.15.92:6827/248263832","cluster_addr":"172.21.15.92:6829/248263832","heartbeat_back_addr":"172.21.15.92:6833/248263832","heartbeat_front_addr":"172.21.15.92:6831/248263832","state":["exists","up"]},{"osd":6,"uuid":"b30eae7c-096a-4d24-ac51-96b8b13ecaee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6811","nonce":4078365985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6813","nonce":4078365985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6817","nonce":4078365985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6815","nonce":4078365985}]},"public_addr":"172.21.15.92:6811/4078365985","cluster_addr":"172.21.15.92:6813/4078365985","heartbeat_back_addr":"172.21.15.92:6817/4078365985","heartbeat_front_addr":"172.21.15.92:6815/4078365985","state":["exists","up"]},{"osd":7,"uuid":"62f12540-d374-4a4c-9ff7-63b7304fe77f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6803","nonce":3444787223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6805","nonce":3444787223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6809","nonce":3444787223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6807","nonce":3444787223}]},"public_addr":"172.21.15.92:6803/3444787223","cluster_addr":"172.21.15.92:6805/3444787223","heartbeat_back_addr":"172.21.15.92:6809/3444787223","heartbeat_front_addr":"172.21.15.92:6807/3444787223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.006420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.111135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.103325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.072656+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.200989+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.060455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.774773+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.865778+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:33:25.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.527+0000 7f914dc5c640 1 -- 172.21.15.37:0/2590520334 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9128078bc0 msgr2=0x7f912807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.527+0000 7f914dc5c640 1 --2- 172.21.15.37:0/2590520334 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9128078bc0 0x7f912807b0d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9130033000 tx=0x7f9130004730 comp rx=0 tx=0).stop 2024-09-17T04:33:25.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.527+0000 7f914dc5c640 1 -- 172.21.15.37:0/2590520334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91480cef40 msgr2=0x7f91481f4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.527+0000 7f914dc5c640 1 --2- 172.21.15.37:0/2590520334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91480cef40 0x7f91481f4c90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f913800c3d0 tx=0x7f913800c890 comp rx=0 tx=0).stop 2024-09-17T04:33:25.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.527+0000 7f914dc5c640 1 -- 172.21.15.37:0/2590520334 shutdown_connections 2024-09-17T04:33:25.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.527+0000 7f914dc5c640 1 --2- 172.21.15.37:0/2590520334 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9128078bc0 0x7f912807b0d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.527+0000 7f914dc5c640 1 --2- 172.21.15.37:0/2590520334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f914815c820 0x7f91480cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.527+0000 7f914dc5c640 1 --2- 172.21.15.37:0/2590520334 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f91480b2c00 0x7f9148160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.527+0000 7f914dc5c640 1 --2- 172.21.15.37:0/2590520334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91480cef40 0x7f91481f4c90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.527+0000 7f914dc5c640 1 -- 172.21.15.37:0/2590520334 >> 172.21.15.37:0/2590520334 conn(0x7f91480d44c0 msgr2=0x7f914815d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:25.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.527+0000 7f914dc5c640 1 -- 172.21.15.37:0/2590520334 shutdown_connections 2024-09-17T04:33:25.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.527+0000 7f914dc5c640 1 -- 172.21.15.37:0/2590520334 wait complete. 2024-09-17T04:33:25.539 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T04:33:25.540 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T04:33:25.540 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T04:33:25.541 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-17T04:33:25.541 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-17T04:33:25.541 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-17T04:33:25.542 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-17T04:33:25.542 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-17T04:33:25.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.765+0000 7fd0f144d640 1 -- 172.21.15.37:0/1785090700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0ec0ce890 msgr2=0x7fd0ec144970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.765+0000 7fd0f144d640 1 --2- 172.21.15.37:0/1785090700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0ec0ce890 0x7fd0ec144970 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc005fd0 tx=0x7fd0dc02a580 comp rx=0 tx=0).stop 2024-09-17T04:33:25.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.765+0000 7fd0f144d640 1 -- 172.21.15.37:0/1785090700 shutdown_connections 2024-09-17T04:33:25.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.765+0000 7fd0f144d640 1 --2- 172.21.15.37:0/1785090700 >> v2:172.21.15.92:3300/0 conn(0x7fd0ec070300 0x7fd0ec070710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.765+0000 7fd0f144d640 1 --2- 172.21.15.37:0/1785090700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0ec0ce890 0x7fd0ec144970 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.765+0000 7fd0f144d640 1 -- 172.21.15.37:0/1785090700 >> 172.21.15.37:0/1785090700 conn(0x7fd0ec06ed20 msgr2=0x7fd0ec06f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:25.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.766+0000 7fd0f144d640 1 -- 172.21.15.37:0/1785090700 shutdown_connections 2024-09-17T04:33:25.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.767+0000 7feee3750640 1 -- 172.21.15.37:0/2046034902 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7feedc073650 msgr2=0x7feedc073a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.767+0000 7feee3750640 1 --2- 172.21.15.37:0/2046034902 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7feedc073650 0x7feedc073a60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feecc005fd0 tx=0x7feecc02a560 comp rx=0 tx=0).stop 2024-09-17T04:33:25.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.767+0000 7feee3750640 1 -- 172.21.15.37:0/2046034902 shutdown_connections 2024-09-17T04:33:25.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.767+0000 7feee3750640 1 --2- 172.21.15.37:0/2046034902 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7feedc073650 0x7feedc073a60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.767+0000 7feee3750640 1 --2- 172.21.15.37:0/2046034902 >> v2:172.21.15.92:3300/0 conn(0x7feedc15b640 0x7feedc0b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.767+0000 7feee3750640 1 -- 172.21.15.37:0/2046034902 >> 172.21.15.37:0/2046034902 conn(0x7feedc0b2790 msgr2=0x7feedc0b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:25.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.766+0000 7fd0f144d640 1 -- 172.21.15.37:0/1785090700 wait complete. 2024-09-17T04:33:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.767+0000 7feee3750640 1 -- 172.21.15.37:0/2046034902 shutdown_connections 2024-09-17T04:33:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.767+0000 7feee3750640 1 -- 172.21.15.37:0/2046034902 wait complete. 2024-09-17T04:33:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.769+0000 7fd0f144d640 1 Processor -- start 2024-09-17T04:33:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.769+0000 7f55c128c640 1 -- 172.21.15.37:0/3249643613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55bc0b4250 msgr2=0x7f55bc0cbef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.769+0000 7f55c128c640 1 --2- 172.21.15.37:0/3249643613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55bc0b4250 0x7f55bc0cbef0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f55bc071aa0 tx=0x7f55b402a780 comp rx=0 tx=0).stop 2024-09-17T04:33:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.769+0000 7fd0f144d640 1 -- start start 2024-09-17T04:33:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.769+0000 7f55c128c640 1 -- 172.21.15.37:0/3249643613 shutdown_connections 2024-09-17T04:33:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.769+0000 7f55c128c640 1 --2- 172.21.15.37:0/3249643613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55bc0b4250 0x7f55bc0cbef0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.769+0000 7f55c128c640 1 --2- 172.21.15.37:0/3249643613 >> v2:172.21.15.92:3300/0 conn(0x7f55bc070c60 0x7f55bc0b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.769+0000 7f55c128c640 1 -- 172.21.15.37:0/3249643613 >> 172.21.15.37:0/3249643613 conn(0x7f55bc06d940 msgr2=0x7f55bc0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:25.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.769+0000 7f55c128c640 1 -- 172.21.15.37:0/3249643613 shutdown_connections 2024-09-17T04:33:25.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.769+0000 7f55c128c640 1 -- 172.21.15.37:0/3249643613 wait complete. 2024-09-17T04:33:25.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.770+0000 7f64e4294640 1 -- 172.21.15.37:0/3831517263 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f64dc070ec0 msgr2=0x7f64dc0703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.770+0000 7f64e4294640 1 --2- 172.21.15.37:0/3831517263 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f64dc070ec0 0x7f64dc0703c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009830 tx=0x7f64cc02a510 comp rx=0 tx=0).stop 2024-09-17T04:33:25.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.770+0000 7f61c1998640 1 -- 172.21.15.37:0/566629069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61b4057590 msgr2=0x7f61b40e3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.770+0000 7f61c1998640 1 --2- 172.21.15.37:0/566629069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61b4057590 0x7f61b40e3680 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f61bc052860 tx=0x7f61bc0a6490 comp rx=0 tx=0).stop 2024-09-17T04:33:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.770+0000 7f64e4294640 1 -- 172.21.15.37:0/3831517263 shutdown_connections 2024-09-17T04:33:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.770+0000 7f64e4294640 1 --2- 172.21.15.37:0/3831517263 >> v2:172.21.15.92:3300/0 conn(0x7f64dc0af030 0x7f64dc0731f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.770+0000 7f64e4294640 1 --2- 172.21.15.37:0/3831517263 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f64dc070ec0 0x7f64dc0703c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.770+0000 7f64e4294640 1 -- 172.21.15.37:0/3831517263 >> 172.21.15.37:0/3831517263 conn(0x7f64dc0d00b0 msgr2=0x7f64dc0d04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.770+0000 7f64e4294640 1 -- 172.21.15.37:0/3831517263 shutdown_connections 2024-09-17T04:33:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.770+0000 7f61c1998640 1 -- 172.21.15.37:0/566629069 shutdown_connections 2024-09-17T04:33:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.770+0000 7f61c1998640 1 --2- 172.21.15.37:0/566629069 >> v2:172.21.15.92:3300/0 conn(0x7f61b40e0c40 0x7f61b40538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.770+0000 7f61c1998640 1 --2- 172.21.15.37:0/566629069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61b4057590 0x7f61b40e3680 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.770+0000 7f61c1998640 1 -- 172.21.15.37:0/566629069 >> 172.21.15.37:0/566629069 conn(0x7f61b4055ea0 msgr2=0x7f61b400e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:25.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.771+0000 7feee3750640 1 Processor -- start 2024-09-17T04:33:25.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.771+0000 7f61c1998640 1 -- 172.21.15.37:0/566629069 shutdown_connections 2024-09-17T04:33:25.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.771+0000 7f61c1998640 1 -- 172.21.15.37:0/566629069 wait complete. 2024-09-17T04:33:25.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.771+0000 7feee3750640 1 -- start start 2024-09-17T04:33:25.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.772+0000 7fd0f144d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0ec070300 0x7fd0ec0b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.772+0000 7fd0f144d640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd0ec0ce890 0x7fd0ec0b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.772+0000 7fd0eaffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0ec070300 0x7fd0ec0b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.772+0000 7fea63d50640 1 -- 172.21.15.37:0/3206776105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea5c070320 msgr2=0x7fea5c0ae0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.772+0000 7fea63d50640 1 --2- 172.21.15.37:0/3206776105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea5c070320 0x7fea5c0ae0c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009880 tx=0x7fea4c02a4f0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.772+0000 7fd0eaffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0ec070300 0x7fd0ec0b13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.37:50622/0 (socket says 172.21.15.37:50622) 2024-09-17T04:33:25.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.772+0000 7fd0f144d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0ec162210 0x7fd0ec0b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.772+0000 7fea63d50640 1 -- 172.21.15.37:0/3206776105 shutdown_connections 2024-09-17T04:33:25.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.772+0000 7fea63d50640 1 --2- 172.21.15.37:0/3206776105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea5c070320 0x7fea5c0ae0c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.772+0000 7fd0eaffd640 1 -- 172.21.15.37:0/3754691637 learned_addr learned my addr 172.21.15.37:0/3754691637 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:25.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.772+0000 7fea63d50640 1 --2- 172.21.15.37:0/3206776105 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fea5c0cb350 0x7fea5c06fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.773+0000 7f55c128c640 1 Processor -- start 2024-09-17T04:33:25.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.773+0000 7fd0ebfff640 1 --2- 172.21.15.37:0/3754691637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0ec162210 0x7fd0ec0b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.772+0000 7fea63d50640 1 -- 172.21.15.37:0/3206776105 >> 172.21.15.37:0/3206776105 conn(0x7fea5c0d00b0 msgr2=0x7fea5c0d04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:25.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.773+0000 7f55c128c640 1 -- start start 2024-09-17T04:33:25.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.773+0000 7fd0f144d640 1 -- 172.21.15.37:0/3754691637 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec0cb880 con 0x7fd0ec162210 2024-09-17T04:33:25.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.771+0000 7f64e4294640 1 -- 172.21.15.37:0/3831517263 wait complete. 2024-09-17T04:33:25.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.773+0000 7fd0f144d640 1 -- 172.21.15.37:0/3754691637 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec0cb700 con 0x7fd0ec070300 2024-09-17T04:33:25.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.773+0000 7fd0f144d640 1 -- 172.21.15.37:0/3754691637 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fd0ec0cba00 con 0x7fd0ec0ce890 2024-09-17T04:33:25.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.773+0000 7f747908b640 1 -- 172.21.15.37:0/1626194967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74740d2390 msgr2=0x7f74740d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.773+0000 7f747908b640 1 --2- 172.21.15.37:0/1626194967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74740d2390 0x7f74740d2b90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f74740d0320 tx=0x7f746c032fe0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.773+0000 7f747908b640 1 -- 172.21.15.37:0/1626194967 shutdown_connections 2024-09-17T04:33:25.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.773+0000 7feee3750640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feedc06f6b0 0x7feedc1e6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.773+0000 7fd0ea7fc640 1 --2- 172.21.15.37:0/3754691637 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd0ec0ce890 0x7fd0ec0b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.773+0000 7f747908b640 1 --2- 172.21.15.37:0/1626194967 >> v2:172.21.15.92:3300/0 conn(0x7f74740d0f30 0x7f7474162710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.773+0000 7f747908b640 1 --2- 172.21.15.37:0/1626194967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74740d2390 0x7f74740d2b90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.773+0000 7f747908b640 1 --2- 172.21.15.37:0/1626194967 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f74740b9380 0x7f74740b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.773+0000 7f747908b640 1 -- 172.21.15.37:0/1626194967 >> 172.21.15.37:0/1626194967 conn(0x7f74740d4280 msgr2=0x7f74740d4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7fd0ebfff640 1 -- 172.21.15.37:0/3754691637 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd0ec0ce890 msgr2=0x7fd0ec0b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7fd0ebfff640 1 --2- 172.21.15.37:0/3754691637 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd0ec0ce890 0x7fd0ec0b5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7fd0ebfff640 1 -- 172.21.15.37:0/3754691637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0ec070300 msgr2=0x7fd0ec0b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7fd0ebfff640 1 --2- 172.21.15.37:0/3754691637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd0ec070300 0x7fd0ec0b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7fd0ebfff640 1 -- 172.21.15.37:0/3754691637 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0ec0b3fd0 con 0x7fd0ec162210 2024-09-17T04:33:25.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7feee3750640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7feedc073650 0x7feedc0c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7fea63d50640 1 -- 172.21.15.37:0/3206776105 shutdown_connections 2024-09-17T04:33:25.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7feee3750640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feedc15b640 0x7feedc0c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7feee3750640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feedc0cf1e0 con 0x7feedc06f6b0 2024-09-17T04:33:25.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7feee3750640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7feedc0cf060 con 0x7feedc15b640 2024-09-17T04:33:25.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7feee3750640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7feedc0cf360 con 0x7feedc073650 2024-09-17T04:33:25.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7fd0ebfff640 1 --2- 172.21.15.37:0/3754691637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0ec162210 0x7fd0ec0b55b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400bdd0 tx=0x7fd0e4009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:25.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7f64e4294640 1 Processor -- start 2024-09-17T04:33:25.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7f61c1998640 1 Processor -- start 2024-09-17T04:33:25.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7f747908b640 1 -- 172.21.15.37:0/1626194967 shutdown_connections 2024-09-17T04:33:25.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7f64e4294640 1 -- start start 2024-09-17T04:33:25.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7feee0ceb640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7feedc073650 0x7feedc0c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7feee0ceb640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7feedc073650 0x7feedc0c8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.37:38682/0 (socket says 172.21.15.37:38682) 2024-09-17T04:33:25.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7fea63d50640 1 -- 172.21.15.37:0/3206776105 wait complete. 2024-09-17T04:33:25.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7feee0ceb640 1 -- 172.21.15.37:0/1994318808 learned_addr learned my addr 172.21.15.37:0/1994318808 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:25.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7feee14ec640 1 --2- 172.21.15.37:0/1994318808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feedc06f6b0 0x7feedc1e6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7feee1ced640 1 --2- 172.21.15.37:0/1994318808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feedc15b640 0x7feedc0c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7fd0d37fe640 1 -- 172.21.15.37:0/3754691637 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4019070 con 0x7fd0ec162210 2024-09-17T04:33:25.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7f747908b640 1 -- 172.21.15.37:0/1626194967 wait complete. 2024-09-17T04:33:25.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7fd0f144d640 1 -- 172.21.15.37:0/3754691637 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0ec0b42b0 con 0x7fd0ec162210 2024-09-17T04:33:25.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7feee0ceb640 1 -- 172.21.15.37:0/1994318808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feedc15b640 msgr2=0x7feedc0c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7feee0ceb640 1 --2- 172.21.15.37:0/1994318808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feedc15b640 0x7feedc0c9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.775+0000 7feee0ceb640 1 -- 172.21.15.37:0/1994318808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feedc06f6b0 msgr2=0x7feedc1e6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.775+0000 7feee0ceb640 1 --2- 172.21.15.37:0/1994318808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feedc06f6b0 0x7feedc1e6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.775+0000 7feee0ceb640 1 -- 172.21.15.37:0/1994318808 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feedc0c9af0 con 0x7feedc073650 2024-09-17T04:33:25.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.775+0000 7feee0ceb640 1 --2- 172.21.15.37:0/1994318808 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7feedc073650 0x7feedc0c8d50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7feecc034040 tx=0x7feecc003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:25.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.775+0000 7f55c128c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55bc070c60 0x7f55bc153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.775+0000 7f55baffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55bc070c60 0x7f55bc153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.776+0000 7f55baffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55bc070c60 0x7f55bc153a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45670/0 (socket says 172.21.15.37:45670) 2024-09-17T04:33:25.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.776+0000 7f55c128c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55bc0b4250 0x7f55bc151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.776+0000 7f55baffd640 1 -- 172.21.15.37:0/4185051469 learned_addr learned my addr 172.21.15.37:0/4185051469 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:25.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.776+0000 7f55ba7fc640 1 --2- 172.21.15.37:0/4185051469 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55bc0b4250 0x7f55bc151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.776+0000 7f55c128c640 1 --2- 172.21.15.37:0/4185051469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55bc0cc430 0x7f55bc152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.775+0000 7fd0d37fe640 1 -- 172.21.15.37:0/3754691637 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0e4014440 con 0x7fd0ec162210 2024-09-17T04:33:25.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.776+0000 7fd0f144d640 1 -- 172.21.15.37:0/3754691637 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0ec0b4870 con 0x7fd0ec162210 2024-09-17T04:33:25.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.776+0000 7fd0d37fe640 1 -- 172.21.15.37:0/3754691637 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e40148b0 con 0x7fd0ec162210 2024-09-17T04:33:25.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.776+0000 7f55c128c640 1 -- 172.21.15.37:0/4185051469 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc1512b0 con 0x7f55bc070c60 2024-09-17T04:33:25.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.776+0000 7f55bb7fe640 1 --2- 172.21.15.37:0/4185051469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55bc0cc430 0x7f55bc152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7f55c128c640 1 -- 172.21.15.37:0/4185051469 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc151130 con 0x7f55bc0cc430 2024-09-17T04:33:25.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7f814767e640 1 -- 172.21.15.37:0/2179653084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81400b53e0 msgr2=0x7f81400ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7f814767e640 1 --2- 172.21.15.37:0/2179653084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81400b53e0 0x7f81400ce510 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8138005fd0 tx=0x7f813802aa30 comp rx=0 tx=0).stop 2024-09-17T04:33:25.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.774+0000 7f61c1998640 1 -- start start 2024-09-17T04:33:25.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7f55c128c640 1 -- 172.21.15.37:0/4185051469 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f55bc151430 con 0x7f55bc0b4250 2024-09-17T04:33:25.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7f814767e640 1 -- 172.21.15.37:0/2179653084 shutdown_connections 2024-09-17T04:33:25.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7f814767e640 1 --2- 172.21.15.37:0/2179653084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81400b53e0 0x7f81400ce510 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7f814767e640 1 --2- 172.21.15.37:0/2179653084 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f81400cd1f0 0x7f81400cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7f814767e640 1 -- 172.21.15.37:0/2179653084 >> 172.21.15.37:0/2179653084 conn(0x7f81400b1ca0 msgr2=0x7f81400b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:25.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7fea63d50640 1 Processor -- start 2024-09-17T04:33:25.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7f61c1998640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61b4053e30 0x7f61b404c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7f61c1998640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61b4057590 0x7f61b417da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7f64e4294640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64dc070ec0 0x7f64dc2020f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7f64e4294640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f64dc0af030 0x7f64dc0ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7f64e4294640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64dc15abd0 0x7f64dc0ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7f64e182f640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f64dc0af030 0x7f64dc0ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7f64e2030640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64dc070ec0 0x7f64dc2020f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7f64e182f640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f64dc0af030 0x7f64dc0ae100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.37:38702/0 (socket says 172.21.15.37:38702) 2024-09-17T04:33:25.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7f64e2030640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64dc070ec0 0x7f64dc2020f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45692/0 (socket says 172.21.15.37:45692) 2024-09-17T04:33:25.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.778+0000 7f64e182f640 1 -- 172.21.15.37:0/2240701451 learned_addr learned my addr 172.21.15.37:0/2240701451 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:25.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7feed1ffb640 1 -- 172.21.15.37:0/1994318808 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc03e030 con 0x7feedc073650 2024-09-17T04:33:25.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7feed1ffb640 1 -- 172.21.15.37:0/1994318808 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feecc004890 con 0x7feedc073650 2024-09-17T04:33:25.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7feee3750640 1 -- 172.21.15.37:0/1994318808 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feedc0c9dd0 con 0x7feedc073650 2024-09-17T04:33:25.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.778+0000 7feee3750640 1 -- 172.21.15.37:0/1994318808 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7feedc0ca390 con 0x7feedc073650 2024-09-17T04:33:25.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.778+0000 7feed1ffb640 1 -- 172.21.15.37:0/1994318808 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc035040 con 0x7feedc073650 2024-09-17T04:33:25.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7f61ba7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61b4057590 0x7f61b417da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7f61ba7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61b4057590 0x7f61b417da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45682/0 (socket says 172.21.15.37:45682) 2024-09-17T04:33:25.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7f61ba7fc640 1 -- 172.21.15.37:0/1435253958 learned_addr learned my addr 172.21.15.37:0/1435253958 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:25.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7f61c1998640 1 --2- 172.21.15.37:0/1435253958 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f61b40e0c40 0x7f61b403a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7f61c1998640 1 -- 172.21.15.37:0/1435253958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f61b4037de0 con 0x7f61b4057590 2024-09-17T04:33:25.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7f61c1998640 1 -- 172.21.15.37:0/1435253958 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f61b4054840 con 0x7f61b4053e30 2024-09-17T04:33:25.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7f61c1998640 1 -- 172.21.15.37:0/1435253958 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f61b4037f60 con 0x7f61b40e0c40 2024-09-17T04:33:25.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7f814767e640 1 -- 172.21.15.37:0/2179653084 shutdown_connections 2024-09-17T04:33:25.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.778+0000 7f64e4294640 1 -- 172.21.15.37:0/2240701451 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc0cae10 con 0x7f64dc070ec0 2024-09-17T04:33:25.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.778+0000 7f64e4294640 1 -- 172.21.15.37:0/2240701451 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc0cac90 con 0x7f64dc15abd0 2024-09-17T04:33:25.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.778+0000 7f64e4294640 1 -- 172.21.15.37:0/2240701451 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f64dc0caf90 con 0x7f64dc0af030 2024-09-17T04:33:25.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.778+0000 7f64e2831640 1 --2- 172.21.15.37:0/2240701451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64dc15abd0 0x7f64dc0ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.778+0000 7f61ba7fc640 1 -- 172.21.15.37:0/1435253958 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f61b40e0c40 msgr2=0x7f61b403a670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T04:33:25.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.778+0000 7f61ba7fc640 1 --2- 172.21.15.37:0/1435253958 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f61b40e0c40 0x7f61b403a670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.778+0000 7f61ba7fc640 1 -- 172.21.15.37:0/1435253958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61b4053e30 msgr2=0x7f61b404c5e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T04:33:25.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.778+0000 7f61ba7fc640 1 --2- 172.21.15.37:0/1435253958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61b4053e30 0x7f61b404c5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.778+0000 7f61ba7fc640 1 -- 172.21.15.37:0/1435253958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b417df50 con 0x7f61b4057590 2024-09-17T04:33:25.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.779+0000 7fd0d37fe640 1 -- 172.21.15.37:0/3754691637 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd0e401c070 con 0x7fd0ec162210 2024-09-17T04:33:25.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.778+0000 7f55ba7fc640 1 -- 172.21.15.37:0/4185051469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55bc0cc430 msgr2=0x7f55bc152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.778+0000 7f55ba7fc640 1 --2- 172.21.15.37:0/4185051469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55bc0cc430 0x7f55bc152350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.778+0000 7f55ba7fc640 1 -- 172.21.15.37:0/4185051469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55bc070c60 msgr2=0x7f55bc153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.778+0000 7f55ba7fc640 1 --2- 172.21.15.37:0/4185051469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55bc070c60 0x7f55bc153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.778+0000 7f55ba7fc640 1 -- 172.21.15.37:0/4185051469 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55bc14a4a0 con 0x7f55bc0b4250 2024-09-17T04:33:25.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.778+0000 7f55ba7fc640 1 --2- 172.21.15.37:0/4185051469 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55bc0b4250 0x7f55bc151e10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f55bc0cf7b0 tx=0x7f55b4032de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:25.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.778+0000 7f55a37fe640 1 -- 172.21.15.37:0/4185051469 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b4041020 con 0x7f55bc0b4250 2024-09-17T04:33:25.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.779+0000 7f55a37fe640 1 -- 172.21.15.37:0/4185051469 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55b4037040 con 0x7f55bc0b4250 2024-09-17T04:33:25.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.779+0000 7f55c128c640 1 -- 172.21.15.37:0/4185051469 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55bc14a740 con 0x7f55bc0b4250 2024-09-17T04:33:25.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.779+0000 7f55c128c640 1 -- 172.21.15.37:0/4185051469 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f55bc14ad00 con 0x7f55bc0b4250 2024-09-17T04:33:25.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.778+0000 7f64e182f640 1 -- 172.21.15.37:0/2240701451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64dc15abd0 msgr2=0x7f64dc0ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.778+0000 7f64e182f640 1 --2- 172.21.15.37:0/2240701451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64dc15abd0 0x7f64dc0ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.778+0000 7f64e182f640 1 -- 172.21.15.37:0/2240701451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64dc070ec0 msgr2=0x7f64dc2020f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.778+0000 7f64e182f640 1 --2- 172.21.15.37:0/2240701451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64dc070ec0 0x7f64dc2020f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.779+0000 7f64e182f640 1 -- 172.21.15.37:0/2240701451 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64dc2063c0 con 0x7f64dc0af030 2024-09-17T04:33:25.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.779+0000 7f64e2030640 1 --2- 172.21.15.37:0/2240701451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64dc070ec0 0x7f64dc2020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:33:25.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.779+0000 7f64e182f640 1 --2- 172.21.15.37:0/2240701451 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f64dc0af030 0x7f64dc0ae100 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f64d800e7c0 tx=0x7f64d800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:25.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.779+0000 7f61ba7fc640 1 --2- 172.21.15.37:0/1435253958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61b4057590 0x7f61b417da10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f61ac00b410 tx=0x7f61ac00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:25.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.779+0000 7fea63d50640 1 -- start start 2024-09-17T04:33:25.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.779+0000 7f64d2ffd640 1 -- 172.21.15.37:0/2240701451 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d800ccc0 con 0x7f64dc0af030 2024-09-17T04:33:25.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.780+0000 7f64d2ffd640 1 -- 172.21.15.37:0/2240701451 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64d8005020 con 0x7f64dc0af030 2024-09-17T04:33:25.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.777+0000 7f814767e640 1 -- 172.21.15.37:0/2179653084 wait complete. 2024-09-17T04:33:25.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.780+0000 7f814767e640 1 Processor -- start 2024-09-17T04:33:25.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.780+0000 7f55a37fe640 1 -- 172.21.15.37:0/4185051469 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b4036070 con 0x7f55bc0b4250 2024-09-17T04:33:25.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.778+0000 7f747908b640 1 Processor -- start 2024-09-17T04:33:25.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.778+0000 7f747908b640 1 -- start start 2024-09-17T04:33:25.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.780+0000 7f747908b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74740b9380 0x7f7474202bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.780+0000 7f747908b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f74740d0f30 0x7f74741fc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.780+0000 7f814767e640 1 -- start start 2024-09-17T04:33:25.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.780+0000 7f64d2ffd640 1 -- 172.21.15.37:0/2240701451 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d800ce60 con 0x7f64dc0af030 2024-09-17T04:33:25.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.780+0000 7f64e4294640 1 -- 172.21.15.37:0/2240701451 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64dc2066a0 con 0x7f64dc0af030 2024-09-17T04:33:25.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.780+0000 7f64e4294640 1 -- 172.21.15.37:0/2240701451 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f64dc206c60 con 0x7f64dc0af030 2024-09-17T04:33:25.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.780+0000 7f747908b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74740d2390 0x7f747415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.781+0000 7f747259c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f74740d0f30 0x7f74741fc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.781+0000 7f747259c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f74740d0f30 0x7f74741fc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.37:38712/0 (socket says 172.21.15.37:38712) 2024-09-17T04:33:25.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.779+0000 7f61a3fff640 1 -- 172.21.15.37:0/1435253958 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac015020 con 0x7f61b4057590 2024-09-17T04:33:25.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.779+0000 7f61a3fff640 1 -- 172.21.15.37:0/1435253958 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61ac011940 con 0x7f61b4057590 2024-09-17T04:33:25.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.781+0000 7feed1ffb640 1 -- 172.21.15.37:0/1994318808 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7feecc041030 con 0x7feedc073650 2024-09-17T04:33:25.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.779+0000 7f61a3fff640 1 -- 172.21.15.37:0/1435253958 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac011d90 con 0x7f61b4057590 2024-09-17T04:33:25.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.781+0000 7f747359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74740d2390 0x7f747415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.781+0000 7f747359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74740d2390 0x7f747415bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45696/0 (socket says 172.21.15.37:45696) 2024-09-17T04:33:25.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.781+0000 7f747259c640 1 -- 172.21.15.37:0/3131770957 learned_addr learned my addr 172.21.15.37:0/3131770957 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:25.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.781+0000 7f747908b640 1 -- 172.21.15.37:0/3131770957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f74740d1340 con 0x7f74740d2390 2024-09-17T04:33:25.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.781+0000 7f747908b640 1 -- 172.21.15.37:0/3131770957 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f74740b52e0 con 0x7f74740b9380 2024-09-17T04:33:25.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.780+0000 7fd0d37fe640 1 --2- 172.21.15.37:0/3754691637 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd0d8078c60 0x7fd0d807b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.780+0000 7fd0d37fe640 1 -- 172.21.15.37:0/3754691637 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd0e409ac10 con 0x7fd0ec162210 2024-09-17T04:33:25.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.780+0000 7fd0f144d640 1 --2- 172.21.15.37:0/3754691637 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x7fd0cc001650 0x7fd0cc003b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.780+0000 7fd0f144d640 1 -- 172.21.15.37:0/3754691637 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd0cc006c00 con 0x7fd0cc001650 2024-09-17T04:33:25.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.781+0000 7fd0ea7fc640 1 --2- 172.21.15.37:0/3754691637 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x7fd0cc001650 0x7fd0cc003b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.782+0000 7fd0eaffd640 1 --2- 172.21.15.37:0/3754691637 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd0d8078c60 0x7fd0d807b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.782+0000 7f64e4294640 1 -- 172.21.15.37:0/2240701451 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f64dc206e60 con 0x7f64dc0af030 2024-09-17T04:33:25.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.781+0000 7feee3750640 1 -- 172.21.15.37:0/1994318808 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feedc0afad0 con 0x7feedc073650 2024-09-17T04:33:25.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.781+0000 7f61c1998640 1 -- 172.21.15.37:0/1435253958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b40d9420 con 0x7f61b4057590 2024-09-17T04:33:25.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.782+0000 7f61c1998640 1 -- 172.21.15.37:0/1435253958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b404d230 con 0x7f61b4057590 2024-09-17T04:33:25.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.782+0000 7feed1ffb640 1 --2- 172.21.15.37:0/1994318808 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7feec0078c60 0x7feec007b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.781+0000 7f747908b640 1 -- 172.21.15.37:0/3131770957 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f74740d14c0 con 0x7f74740d0f30 2024-09-17T04:33:25.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.782+0000 7f55a37fe640 1 -- 172.21.15.37:0/4185051469 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f55b4044020 con 0x7f55bc0b4250 2024-09-17T04:33:25.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.782+0000 7feed1ffb640 1 -- 172.21.15.37:0/1994318808 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7feecc0bb880 con 0x7feedc073650 2024-09-17T04:33:25.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.782+0000 7f7472d9d640 1 --2- 172.21.15.37:0/3131770957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74740b9380 0x7f7474202bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.781+0000 7fea63d50640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea5c070320 0x7fea5c1f9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.783+0000 7fd0ea7fc640 1 --2- 172.21.15.37:0/3754691637 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x7fd0cc001650 0x7fd0cc003b60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:25.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.783+0000 7f55a37fe640 1 --2- 172.21.15.37:0/4185051469 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f55a8078b90 0x7f55a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.783+0000 7f747259c640 1 -- 172.21.15.37:0/3131770957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74740b9380 msgr2=0x7f7474202bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.783+0000 7f747259c640 1 --2- 172.21.15.37:0/3131770957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74740b9380 0x7f7474202bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.783+0000 7f747259c640 1 -- 172.21.15.37:0/3131770957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74740d2390 msgr2=0x7f747415bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.783+0000 7f747259c640 1 --2- 172.21.15.37:0/3131770957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74740d2390 0x7f747415bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.783+0000 7f747259c640 1 -- 172.21.15.37:0/3131770957 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7474203100 con 0x7f74740d0f30 2024-09-17T04:33:25.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.781+0000 7fea63d50640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fea5c0cdfb0 0x7fea5c0b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.782+0000 7fea63d50640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea5c1f9f70 0x7fea5c0b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.782+0000 7fea63d50640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c0ae600 con 0x7fea5c1f9f70 2024-09-17T04:33:25.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.782+0000 7fea63d50640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c070ab0 con 0x7fea5c070320 2024-09-17T04:33:25.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.782+0000 7fea63d50640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fea5c0ae780 con 0x7fea5c0cdfb0 2024-09-17T04:33:25.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.782+0000 7fea61aec640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea5c070320 0x7fea5c1f9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.782+0000 7fea61aec640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea5c070320 0x7fea5c1f9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.37:50672/0 (socket says 172.21.15.37:50672) 2024-09-17T04:33:25.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.782+0000 7fea61aec640 1 -- 172.21.15.37:0/3252681947 learned_addr learned my addr 172.21.15.37:0/3252681947 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:25.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.782+0000 7fea61aec640 1 -- 172.21.15.37:0/3252681947 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fea5c0cdfb0 msgr2=0x7fea5c0b3a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T04:33:25.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.782+0000 7fea61aec640 1 --2- 172.21.15.37:0/3252681947 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fea5c0cdfb0 0x7fea5c0b3a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.782+0000 7fea61aec640 1 -- 172.21.15.37:0/3252681947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea5c1f9f70 msgr2=0x7fea5c0b3f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T04:33:25.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.782+0000 7fea61aec640 1 --2- 172.21.15.37:0/3252681947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea5c1f9f70 0x7fea5c0b3f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.783+0000 7fea61aec640 1 -- 172.21.15.37:0/3252681947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5c1fa6b0 con 0x7fea5c070320 2024-09-17T04:33:25.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.783+0000 7f55c128c640 1 -- 172.21.15.37:0/4185051469 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f55bc0008d0 con 0x7f55bc0b4250 2024-09-17T04:33:25.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.783+0000 7fea61aec640 1 --2- 172.21.15.37:0/3252681947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea5c070320 0x7fea5c1f9a30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fea5400caa0 tx=0x7fea5400cea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:25.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.782+0000 7feee14ec640 1 --2- 172.21.15.37:0/1994318808 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7feec0078c60 0x7feec007b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.783+0000 7feed1ffb640 1 --2- 172.21.15.37:0/1994318808 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x7feec0080f30 0x7feec0083380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.783+0000 7feed1ffb640 1 -- 172.21.15.37:0/1994318808 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feecc005310 con 0x7feec0080f30 2024-09-17T04:33:25.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.783+0000 7feed1ffb640 1 -- 172.21.15.37:0/1994318808 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7feecc0c2050 con 0x7feedc073650 2024-09-17T04:33:25.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.783+0000 7f747259c640 1 --2- 172.21.15.37:0/3131770957 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f74740d0f30 0x7f74741fc070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f746c0335d0 tx=0x7f746c008a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:25.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.783+0000 7f55baffd640 1 --2- 172.21.15.37:0/4185051469 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f55a8078b90 0x7f55a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.783+0000 7f55a37fe640 1 -- 172.21.15.37:0/4185051469 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f55b40bb8b0 con 0x7f55bc0b4250 2024-09-17T04:33:25.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.783+0000 7f64d2ffd640 1 -- 172.21.15.37:0/2240701451 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f64d8005470 con 0x7f64dc0af030 2024-09-17T04:33:25.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.783+0000 7feee14ec640 1 --2- 172.21.15.37:0/1994318808 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7feec0078c60 0x7feec007b170 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7feed8006fd0 tx=0x7feed800f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:25.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.783+0000 7f745b7fe640 1 -- 172.21.15.37:0/3131770957 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c013020 con 0x7f74740d0f30 2024-09-17T04:33:25.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.783+0000 7feee1ced640 1 --2- 172.21.15.37:0/1994318808 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x7feec0080f30 0x7feec0083380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.783+0000 7f55a37fe640 1 --2- 172.21.15.37:0/4185051469 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x7f55a8080de0 0x7f55a8083230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.783+0000 7f745b7fe640 1 -- 172.21.15.37:0/3131770957 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f746c0132d0 con 0x7f74740d0f30 2024-09-17T04:33:25.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.783+0000 7f745b7fe640 1 -- 172.21.15.37:0/3131770957 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c004040 con 0x7f74740d0f30 2024-09-17T04:33:25.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.784+0000 7f747908b640 1 -- 172.21.15.37:0/3131770957 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74741fc5b0 con 0x7f74740d0f30 2024-09-17T04:33:25.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.784+0000 7fea527fc640 1 -- 172.21.15.37:0/3252681947 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54016020 con 0x7fea5c070320 2024-09-17T04:33:25.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.784+0000 7f747908b640 1 -- 172.21.15.37:0/3131770957 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f74740c7200 con 0x7f74740d0f30 2024-09-17T04:33:25.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.784+0000 7fea527fc640 1 -- 172.21.15.37:0/3252681947 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea5400ab10 con 0x7fea5c070320 2024-09-17T04:33:25.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.784+0000 7fea63d50640 1 -- 172.21.15.37:0/3252681947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5c0c5e70 con 0x7fea5c070320 2024-09-17T04:33:25.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.784+0000 7fea63d50640 1 -- 172.21.15.37:0/3252681947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea5c0c5b60 con 0x7fea5c070320 2024-09-17T04:33:25.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.784+0000 7fd0d37fe640 1 -- 172.21.15.37:0/3754691637 <== osd.1 v2:172.21.15.37:6824/2657955935 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd0cc006c00 con 0x7fd0cc001650 2024-09-17T04:33:25.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.785+0000 7fd0eaffd640 1 --2- 172.21.15.37:0/3754691637 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd0d8078c60 0x7fd0d807b170 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009230 tx=0x7fd0dc004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:25.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.785+0000 7f55baffd640 1 --2- 172.21.15.37:0/4185051469 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f55a8078b90 0x7f55a807b0a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0099a0 tx=0x7f55ac006f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:25.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.785+0000 7f55a37fe640 1 -- 172.21.15.37:0/4185051469 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f55b4033ec0 con 0x7f55a8080de0 2024-09-17T04:33:25.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.784+0000 7f64d2ffd640 1 --2- 172.21.15.37:0/2240701451 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f64c0078c90 0x7f64c007b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.784+0000 7f64d2ffd640 1 -- 172.21.15.37:0/2240701451 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f64d809b4a0 con 0x7f64dc0af030 2024-09-17T04:33:25.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.784+0000 7f64e2030640 1 --2- 172.21.15.37:0/2240701451 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f64c0078c90 0x7f64c007b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.785+0000 7f64d2ffd640 1 --2- 172.21.15.37:0/2240701451 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x7f64c0080f60 0x7f64c00833b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.785+0000 7f64d2ffd640 1 -- 172.21.15.37:0/2240701451 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f64c0083a80 con 0x7f64c0080f60 2024-09-17T04:33:25.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.784+0000 7f814767e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81400b49a0 0x7f814009c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.784+0000 7f814767e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81400b53e0 0x7f81400ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.784+0000 7f814767e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f81400ad670 0x7f81400ada80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.784+0000 7f814767e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f81400cea50 con 0x7f81400b49a0 2024-09-17T04:33:25.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.784+0000 7f814767e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8140070cc0 con 0x7f81400b53e0 2024-09-17T04:33:25.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.784+0000 7f814767e640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f81400cebd0 con 0x7f81400ad670 2024-09-17T04:33:25.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.784+0000 7f8145c1b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f81400ad670 0x7f81400ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.784+0000 7f8145c1b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f81400ad670 0x7f81400ada80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.37:38718/0 (socket says 172.21.15.37:38718) 2024-09-17T04:33:25.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.784+0000 7f8145c1b640 1 -- 172.21.15.37:0/495517500 learned_addr learned my addr 172.21.15.37:0/495517500 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:25.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.785+0000 7f55a37fe640 1 -- 172.21.15.37:0/4185051469 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f55b407f4a0 con 0x7f55bc0b4250 2024-09-17T04:33:25.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.785+0000 7f64d2ffd640 1 -- 172.21.15.37:0/2240701451 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f64d80a2050 con 0x7f64dc0af030 2024-09-17T04:33:25.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.785+0000 7f55bb7fe640 1 --2- 172.21.15.37:0/4185051469 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x7f55a8080de0 0x7f55a8083230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.785+0000 7f64e2030640 1 --2- 172.21.15.37:0/2240701451 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f64c0078c90 0x7f64c007b1a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f64cc034000 tx=0x7f64cc004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:25.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.785+0000 7fea527fc640 1 -- 172.21.15.37:0/3252681947 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54012420 con 0x7fea5c070320 2024-09-17T04:33:25.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.786+0000 7f61a3fff640 1 -- 172.21.15.37:0/1435253958 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f61ac018020 con 0x7f61b4057590 2024-09-17T04:33:25.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.786+0000 7fea63d50640 1 -- 172.21.15.37:0/3252681947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fea5c0aec50 con 0x7fea5c070320 2024-09-17T04:33:25.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.786+0000 7f64e2831640 1 --2- 172.21.15.37:0/2240701451 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x7f64c0080f60 0x7f64c00833b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.786+0000 7feee1ced640 1 --2- 172.21.15.37:0/1994318808 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x7feec0080f30 0x7feec0083380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:25.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.786+0000 7f745b7fe640 1 -- 172.21.15.37:0/3131770957 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f746c042020 con 0x7f74740d0f30 2024-09-17T04:33:25.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.787+0000 7f61a3fff640 1 --2- 172.21.15.37:0/1435253958 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f61a8078c60 0x7f61a807b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.787+0000 7f61a3fff640 1 -- 172.21.15.37:0/1435253958 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f61ac09aad0 con 0x7f61b4057590 2024-09-17T04:33:25.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.787+0000 7f55bb7fe640 1 --2- 172.21.15.37:0/4185051469 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x7f55a8080de0 0x7f55a8083230 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:25.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.787+0000 7f64e2831640 1 --2- 172.21.15.37:0/2240701451 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x7f64c0080f60 0x7f64c00833b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:25.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.786+0000 7f8144c19640 1 --2- 172.21.15.37:0/495517500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81400b53e0 0x7f81400ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.786+0000 7f8144c19640 1 -- 172.21.15.37:0/495517500 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f81400ad670 msgr2=0x7f81400ada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.786+0000 7f8144c19640 1 --2- 172.21.15.37:0/495517500 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f81400ad670 0x7f81400ada80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.786+0000 7f8144c19640 1 -- 172.21.15.37:0/495517500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81400b49a0 msgr2=0x7f814009c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.786+0000 7f814541a640 1 --2- 172.21.15.37:0/495517500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81400b49a0 0x7f814009c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.786+0000 7f8144c19640 1 --2- 172.21.15.37:0/495517500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81400b49a0 0x7f814009c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.786+0000 7f8144c19640 1 -- 172.21.15.37:0/495517500 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81400adfc0 con 0x7f81400b53e0 2024-09-17T04:33:25.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.786+0000 7f814541a640 1 --2- 172.21.15.37:0/495517500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81400b49a0 0x7f814009c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:25.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.787+0000 7f8144c19640 1 --2- 172.21.15.37:0/495517500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81400b53e0 0x7f81400ad130 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f813000b3e0 tx=0x7f813000b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:25.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.787+0000 7f61c1998640 1 --2- 172.21.15.37:0/1435253958 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x7f61b40d8e40 0x7f61b403c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.787+0000 7f8135ffb640 1 -- 172.21.15.37:0/495517500 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8130015020 con 0x7f81400b53e0 2024-09-17T04:33:25.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.787+0000 7f8135ffb640 1 -- 172.21.15.37:0/495517500 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8130011940 con 0x7f81400b53e0 2024-09-17T04:33:25.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.787+0000 7f8135ffb640 1 -- 172.21.15.37:0/495517500 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8130011d90 con 0x7f81400b53e0 2024-09-17T04:33:25.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.787+0000 7f747908b640 1 -- 172.21.15.37:0/3131770957 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7454001030 con 0x7f74740d0f30 2024-09-17T04:33:25.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.787+0000 7f745b7fe640 1 --2- 172.21.15.37:0/3131770957 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f7460078c10 0x7f746007b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.787+0000 7feed1ffb640 1 -- 172.21.15.37:0/1994318808 <== osd.4 v2:172.21.15.92:6818/2509785752 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feecc005310 con 0x7feec0080f30 2024-09-17T04:33:25.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.787+0000 7f745b7fe640 1 -- 172.21.15.37:0/3131770957 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f746c0bbd80 con 0x7f74740d0f30 2024-09-17T04:33:25.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.787+0000 7f7472d9d640 1 --2- 172.21.15.37:0/3131770957 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f7460078c10 0x7f746007b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.787+0000 7f61c1998640 1 -- 172.21.15.37:0/1435253958 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f61b404cce0 con 0x7f61b40d8e40 2024-09-17T04:33:25.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.787+0000 7f61baffd640 1 --2- 172.21.15.37:0/1435253958 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f61a8078c60 0x7f61a807b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.787+0000 7f814767e640 1 -- 172.21.15.37:0/495517500 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f814009cf80 con 0x7f81400b53e0 2024-09-17T04:33:25.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.787+0000 7f745b7fe640 1 --2- 172.21.15.37:0/3131770957 >> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] conn(0x7f7460080e00 0x7f7460083250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.788+0000 7f814767e640 1 -- 172.21.15.37:0/495517500 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f814009d540 con 0x7f81400b53e0 2024-09-17T04:33:25.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.788+0000 7f745b7fe640 1 -- 172.21.15.37:0/3131770957 --> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f746c045e50 con 0x7f7460080e00 2024-09-17T04:33:25.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.788+0000 7f747359e640 1 --2- 172.21.15.37:0/3131770957 >> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] conn(0x7f7460080e00 0x7f7460083250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.788+0000 7f7472d9d640 1 --2- 172.21.15.37:0/3131770957 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f7460078c10 0x7f746007b120 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7464005c60 tx=0x7f7464005bf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:25.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.787+0000 7fea527fc640 1 -- 172.21.15.37:0/3252681947 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fea54019020 con 0x7fea5c070320 2024-09-17T04:33:25.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.788+0000 7f745b7fe640 1 -- 172.21.15.37:0/3131770957 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f746c07fb50 con 0x7f74740d0f30 2024-09-17T04:33:25.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.788+0000 7f55a37fe640 1 -- 172.21.15.37:0/4185051469 <== osd.2 v2:172.21.15.37:6816/579720176 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f55b4033ec0 con 0x7f55a8080de0 2024-09-17T04:33:25.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.788+0000 7f64d2ffd640 1 -- 172.21.15.37:0/2240701451 <== osd.3 v2:172.21.15.37:6800/809975826 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f64c0083a80 con 0x7f64c0080f60 2024-09-17T04:33:25.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.788+0000 7f747359e640 1 --2- 172.21.15.37:0/3131770957 >> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] conn(0x7f7460080e00 0x7f7460083250 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:25.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.788+0000 7f61baffd640 1 --2- 172.21.15.37:0/1435253958 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f61a8078c60 0x7f61a807b170 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f61b4056de0 tx=0x7f61bc072040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:25.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.789+0000 7f61bb7fe640 1 --2- 172.21.15.37:0/1435253958 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x7f61b40d8e40 0x7f61b403c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.789+0000 7f814767e640 1 -- 172.21.15.37:0/495517500 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f81400cf0a0 con 0x7f81400b53e0 2024-09-17T04:33:25.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.789+0000 7fea527fc640 1 --2- 172.21.15.37:0/3252681947 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fea40078c60 0x7fea4007b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.789+0000 7fea612eb640 1 --2- 172.21.15.37:0/3252681947 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fea40078c60 0x7fea4007b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.789+0000 7f61bb7fe640 1 --2- 172.21.15.37:0/1435253958 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x7f61b40d8e40 0x7f61b403c230 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:25.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.789+0000 7fea527fc640 1 -- 172.21.15.37:0/3252681947 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fea5409b150 con 0x7fea5c070320 2024-09-17T04:33:25.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.789+0000 7f745b7fe640 1 -- 172.21.15.37:0/3131770957 <== osd.6 v2:172.21.15.92:6810/4078365985 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f746c045e50 con 0x7f7460080e00 2024-09-17T04:33:25.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.790+0000 7fea612eb640 1 --2- 172.21.15.37:0/3252681947 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fea40078c60 0x7fea4007b170 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fea5c06f110 tx=0x7fea58009260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:25.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.790+0000 7fea527fc640 1 --2- 172.21.15.37:0/3252681947 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x7fea40080f30 0x7fea40083380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.790+0000 7fea527fc640 1 -- 172.21.15.37:0/3252681947 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fea5401fe70 con 0x7fea40080f30 2024-09-17T04:33:25.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.790+0000 7fea527fc640 1 -- 172.21.15.37:0/3252681947 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fea540a2050 con 0x7fea5c070320 2024-09-17T04:33:25.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.790+0000 7fea622ed640 1 --2- 172.21.15.37:0/3252681947 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x7fea40080f30 0x7fea40083380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.791+0000 7f61a3fff640 1 -- 172.21.15.37:0/1435253958 <== osd.5 v2:172.21.15.92:6826/248263832 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f61b404cce0 con 0x7f61b40d8e40 2024-09-17T04:33:25.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.791+0000 7fea622ed640 1 --2- 172.21.15.37:0/3252681947 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x7fea40080f30 0x7fea40083380 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:25.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.791+0000 7f8135ffb640 1 -- 172.21.15.37:0/495517500 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8130018020 con 0x7f81400b53e0 2024-09-17T04:33:25.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.791+0000 7f8135ffb640 1 --2- 172.21.15.37:0/495517500 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8124078be0 0x7f812407b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.791+0000 7f814541a640 1 --2- 172.21.15.37:0/495517500 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8124078be0 0x7f812407b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.791+0000 7f8135ffb640 1 -- 172.21.15.37:0/495517500 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f813005f150 con 0x7f81400b53e0 2024-09-17T04:33:25.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.791+0000 7f8135ffb640 1 --2- 172.21.15.37:0/495517500 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x7f8124081a50 0x7f8124083f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:25.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.791+0000 7f8135ffb640 1 -- 172.21.15.37:0/495517500 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8124084610 con 0x7f8124081a50 2024-09-17T04:33:25.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.792+0000 7f8135ffb640 1 -- 172.21.15.37:0/495517500 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f813005f6d0 con 0x7f81400b53e0 2024-09-17T04:33:25.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.792+0000 7f8145c1b640 1 --2- 172.21.15.37:0/495517500 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x7f8124081a50 0x7f8124083f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:25.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.792+0000 7f814541a640 1 --2- 172.21.15.37:0/495517500 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8124078be0 0x7f812407b0f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f813c007fd0 tx=0x7f813c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:25.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.792+0000 7f8145c1b640 1 --2- 172.21.15.37:0/495517500 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x7f8124081a50 0x7f8124083f40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:25.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.792+0000 7fea527fc640 1 -- 172.21.15.37:0/3252681947 <== osd.0 v2:172.21.15.37:6808/2283587593 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fea5401fe70 con 0x7fea40080f30 2024-09-17T04:33:25.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.793+0000 7f8135ffb640 1 -- 172.21.15.37:0/495517500 <== osd.7 v2:172.21.15.92:6802/3444787223 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8124084610 con 0x7f8124081a50 2024-09-17T04:33:25.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.815+0000 7fd0f144d640 1 -- 172.21.15.37:0/3754691637 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd0cc005ce0 con 0x7fd0cc001650 2024-09-17T04:33:25.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.816+0000 7fd0d37fe640 1 -- 172.21.15.37:0/3754691637 <== osd.1 v2:172.21.15.37:6824/2657955935 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd0cc005ce0 con 0x7fd0cc001650 2024-09-17T04:33:25.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.818+0000 7fd0d17fa640 1 -- 172.21.15.37:0/3754691637 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x7fd0cc001650 msgr2=0x7fd0cc003b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.818+0000 7fd0d17fa640 1 --2- 172.21.15.37:0/3754691637 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x7fd0cc001650 0x7fd0cc003b60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.818+0000 7fd0d17fa640 1 -- 172.21.15.37:0/3754691637 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd0d8078c60 msgr2=0x7fd0d807b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.818+0000 7fd0d17fa640 1 --2- 172.21.15.37:0/3754691637 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd0d8078c60 0x7fd0d807b170 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009230 tx=0x7fd0dc004730 comp rx=0 tx=0).stop 2024-09-17T04:33:25.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.818+0000 7fd0d17fa640 1 -- 172.21.15.37:0/3754691637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0ec162210 msgr2=0x7fd0ec0b55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.818+0000 7fd0d17fa640 1 --2- 172.21.15.37:0/3754691637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0ec162210 0x7fd0ec0b55b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400bdd0 tx=0x7fd0e4009ed0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.818+0000 7fd0ebfff640 1 -- 172.21.15.37:0/3754691637 reap_dead start 2024-09-17T04:33:25.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.818+0000 7fd0d17fa640 1 -- 172.21.15.37:0/3754691637 shutdown_connections 2024-09-17T04:33:25.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.818+0000 7fd0d17fa640 1 -- 172.21.15.37:0/3754691637 >> 172.21.15.37:0/3754691637 conn(0x7fd0ec06ed20 msgr2=0x7fd0ec0744c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:25.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.818+0000 7f64e4294640 1 -- 172.21.15.37:0/2240701451 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f64dc206170 con 0x7f64c0080f60 2024-09-17T04:33:25.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.819+0000 7fd0d17fa640 1 -- 172.21.15.37:0/3754691637 shutdown_connections 2024-09-17T04:33:25.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.819+0000 7fd0d17fa640 1 -- 172.21.15.37:0/3754691637 wait complete. 2024-09-17T04:33:25.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.819+0000 7f64d2ffd640 1 -- 172.21.15.37:0/2240701451 <== osd.3 v2:172.21.15.37:6800/809975826 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f64dc206170 con 0x7f64c0080f60 2024-09-17T04:33:25.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.820+0000 7f64e4294640 1 -- 172.21.15.37:0/2240701451 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x7f64c0080f60 msgr2=0x7f64c00833b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.820+0000 7f64e4294640 1 --2- 172.21.15.37:0/2240701451 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x7f64c0080f60 0x7f64c00833b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.820+0000 7f55c128c640 1 -- 172.21.15.37:0/4185051469 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f55bc153f50 con 0x7f55a8080de0 2024-09-17T04:33:25.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.820+0000 7f64e4294640 1 -- 172.21.15.37:0/2240701451 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f64c0078c90 msgr2=0x7f64c007b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.820+0000 7f64e4294640 1 --2- 172.21.15.37:0/2240701451 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f64c0078c90 0x7f64c007b1a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f64cc034000 tx=0x7f64cc004730 comp rx=0 tx=0).stop 2024-09-17T04:33:25.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.820+0000 7f64e4294640 1 -- 172.21.15.37:0/2240701451 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f64dc0af030 msgr2=0x7f64dc0ae100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.820+0000 7f64e4294640 1 --2- 172.21.15.37:0/2240701451 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f64dc0af030 0x7f64dc0ae100 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f64d800e7c0 tx=0x7f64d800ec80 comp rx=0 tx=0).stop 2024-09-17T04:33:25.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.820+0000 7f64e2831640 1 -- 172.21.15.37:0/2240701451 reap_dead start 2024-09-17T04:33:25.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.820+0000 7f64e4294640 1 -- 172.21.15.37:0/2240701451 shutdown_connections 2024-09-17T04:33:25.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.820+0000 7f64e4294640 1 -- 172.21.15.37:0/2240701451 >> 172.21.15.37:0/2240701451 conn(0x7f64dc0d00b0 msgr2=0x7f64dc1eaa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:25.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.820+0000 7feee3750640 1 -- 172.21.15.37:0/1994318808 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feedc0c9880 con 0x7feec0080f30 2024-09-17T04:33:25.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.821+0000 7f64e4294640 1 -- 172.21.15.37:0/2240701451 shutdown_connections 2024-09-17T04:33:25.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.821+0000 7f64e4294640 1 -- 172.21.15.37:0/2240701451 wait complete. 2024-09-17T04:33:25.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.821+0000 7f55a37fe640 1 -- 172.21.15.37:0/4185051469 <== osd.2 v2:172.21.15.37:6816/579720176 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f55bc153f50 con 0x7f55a8080de0 2024-09-17T04:33:25.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.821+0000 7feed1ffb640 1 -- 172.21.15.37:0/1994318808 <== osd.4 v2:172.21.15.92:6818/2509785752 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7feedc0c9880 con 0x7feec0080f30 2024-09-17T04:33:25.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.821+0000 7f747908b640 1 -- 172.21.15.37:0/3131770957 --> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7454002dd0 con 0x7f7460080e00 2024-09-17T04:33:25.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.821+0000 7f55c128c640 1 -- 172.21.15.37:0/4185051469 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x7f55a8080de0 msgr2=0x7f55a8083230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.821+0000 7f55c128c640 1 --2- 172.21.15.37:0/4185051469 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x7f55a8080de0 0x7f55a8083230 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.822+0000 7f55c128c640 1 -- 172.21.15.37:0/4185051469 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f55a8078b90 msgr2=0x7f55a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.822+0000 7f55c128c640 1 --2- 172.21.15.37:0/4185051469 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f55a8078b90 0x7f55a807b0a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0099a0 tx=0x7f55ac006f90 comp rx=0 tx=0).stop 2024-09-17T04:33:25.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.822+0000 7f55c128c640 1 -- 172.21.15.37:0/4185051469 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55bc0b4250 msgr2=0x7f55bc151e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.822+0000 7f55c128c640 1 --2- 172.21.15.37:0/4185051469 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55bc0b4250 0x7f55bc151e10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f55bc0cf7b0 tx=0x7f55b4032de0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.822+0000 7f55bb7fe640 1 -- 172.21.15.37:0/4185051469 reap_dead start 2024-09-17T04:33:25.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.822+0000 7f745b7fe640 1 -- 172.21.15.37:0/3131770957 <== osd.6 v2:172.21.15.92:6810/4078365985 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7454002dd0 con 0x7f7460080e00 2024-09-17T04:33:25.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.822+0000 7feee3750640 1 -- 172.21.15.37:0/1994318808 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x7feec0080f30 msgr2=0x7feec0083380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.822+0000 7feee3750640 1 --2- 172.21.15.37:0/1994318808 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x7feec0080f30 0x7feec0083380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.822+0000 7f55c128c640 1 -- 172.21.15.37:0/4185051469 shutdown_connections 2024-09-17T04:33:25.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.822+0000 7f55c128c640 1 -- 172.21.15.37:0/4185051469 >> 172.21.15.37:0/4185051469 conn(0x7f55bc06d940 msgr2=0x7f55bc0ce6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:25.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.822+0000 7f747908b640 1 -- 172.21.15.37:0/3131770957 >> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] conn(0x7f7460080e00 msgr2=0x7f7460083250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.822+0000 7f747908b640 1 --2- 172.21.15.37:0/3131770957 >> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] conn(0x7f7460080e00 0x7f7460083250 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.822+0000 7f61c1998640 1 -- 172.21.15.37:0/1435253958 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f61b40dc580 con 0x7f61b40d8e40 2024-09-17T04:33:25.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.822+0000 7f55c128c640 1 -- 172.21.15.37:0/4185051469 shutdown_connections 2024-09-17T04:33:25.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.822+0000 7f55c128c640 1 -- 172.21.15.37:0/4185051469 wait complete. 2024-09-17T04:33:25.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.823+0000 7f747908b640 1 -- 172.21.15.37:0/3131770957 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f7460078c10 msgr2=0x7f746007b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.823+0000 7f747908b640 1 --2- 172.21.15.37:0/3131770957 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f7460078c10 0x7f746007b120 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7464005c60 tx=0x7f7464005bf0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.823+0000 7f747908b640 1 -- 172.21.15.37:0/3131770957 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f74740d0f30 msgr2=0x7f74741fc070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.823+0000 7f747908b640 1 --2- 172.21.15.37:0/3131770957 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f74740d0f30 0x7f74741fc070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f746c0335d0 tx=0x7f746c008a20 comp rx=0 tx=0).stop 2024-09-17T04:33:25.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.823+0000 7f747359e640 1 -- 172.21.15.37:0/3131770957 reap_dead start 2024-09-17T04:33:25.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.823+0000 7f747908b640 1 -- 172.21.15.37:0/3131770957 shutdown_connections 2024-09-17T04:33:25.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.823+0000 7f747908b640 1 -- 172.21.15.37:0/3131770957 >> 172.21.15.37:0/3131770957 conn(0x7f74740d4280 msgr2=0x7f747406d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:25.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.823+0000 7f747908b640 1 -- 172.21.15.37:0/3131770957 shutdown_connections 2024-09-17T04:33:25.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.823+0000 7feee3750640 1 -- 172.21.15.37:0/1994318808 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7feec0078c60 msgr2=0x7feec007b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.823+0000 7feee3750640 1 --2- 172.21.15.37:0/1994318808 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7feec0078c60 0x7feec007b170 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7feed8006fd0 tx=0x7feed800f040 comp rx=0 tx=0).stop 2024-09-17T04:33:25.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.823+0000 7f747908b640 1 -- 172.21.15.37:0/3131770957 wait complete. 2024-09-17T04:33:25.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.823+0000 7f61a3fff640 1 -- 172.21.15.37:0/1435253958 <== osd.5 v2:172.21.15.92:6826/248263832 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f61b404cce0 con 0x7f61b40d8e40 2024-09-17T04:33:25.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.823+0000 7feee3750640 1 -- 172.21.15.37:0/1994318808 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7feedc073650 msgr2=0x7feedc0c8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.823+0000 7feee3750640 1 --2- 172.21.15.37:0/1994318808 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7feedc073650 0x7feedc0c8d50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7feecc034040 tx=0x7feecc003c10 comp rx=0 tx=0).stop 2024-09-17T04:33:25.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.823+0000 7feee1ced640 1 -- 172.21.15.37:0/1994318808 reap_dead start 2024-09-17T04:33:25.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.824+0000 7feee3750640 1 -- 172.21.15.37:0/1994318808 shutdown_connections 2024-09-17T04:33:25.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.824+0000 7feee3750640 1 -- 172.21.15.37:0/1994318808 >> 172.21.15.37:0/1994318808 conn(0x7feedc0b2790 msgr2=0x7feedc06fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:25.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.824+0000 7f61c1998640 1 -- 172.21.15.37:0/1435253958 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x7f61b40d8e40 msgr2=0x7f61b403c230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.824+0000 7f61c1998640 1 --2- 172.21.15.37:0/1435253958 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x7f61b40d8e40 0x7f61b403c230 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.824+0000 7feee3750640 1 -- 172.21.15.37:0/1994318808 shutdown_connections 2024-09-17T04:33:25.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.824+0000 7feee3750640 1 -- 172.21.15.37:0/1994318808 wait complete. 2024-09-17T04:33:25.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.824+0000 7f61c1998640 1 -- 172.21.15.37:0/1435253958 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f61a8078c60 msgr2=0x7f61a807b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.824+0000 7f61c1998640 1 --2- 172.21.15.37:0/1435253958 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f61a8078c60 0x7f61a807b170 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f61b4056de0 tx=0x7f61bc072040 comp rx=0 tx=0).stop 2024-09-17T04:33:25.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.824+0000 7f61c1998640 1 -- 172.21.15.37:0/1435253958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61b4057590 msgr2=0x7f61b417da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.824+0000 7f61c1998640 1 --2- 172.21.15.37:0/1435253958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61b4057590 0x7f61b417da10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f61ac00b410 tx=0x7f61ac00b8d0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.824+0000 7f61bb7fe640 1 -- 172.21.15.37:0/1435253958 reap_dead start 2024-09-17T04:33:25.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.824+0000 7f61c1998640 1 -- 172.21.15.37:0/1435253958 shutdown_connections 2024-09-17T04:33:25.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.824+0000 7f61c1998640 1 -- 172.21.15.37:0/1435253958 >> 172.21.15.37:0/1435253958 conn(0x7f61b4055ea0 msgr2=0x7f61b40dedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:25.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.825+0000 7fea63d50640 1 -- 172.21.15.37:0/3252681947 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fea5c06a110 con 0x7fea40080f30 2024-09-17T04:33:25.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.825+0000 7f61c1998640 1 -- 172.21.15.37:0/1435253958 shutdown_connections 2024-09-17T04:33:25.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.825+0000 7f61c1998640 1 -- 172.21.15.37:0/1435253958 wait complete. 2024-09-17T04:33:25.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.825+0000 7f814767e640 1 -- 172.21.15.37:0/495517500 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f814009ac20 con 0x7f8124081a50 2024-09-17T04:33:25.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.825+0000 7fea527fc640 1 -- 172.21.15.37:0/3252681947 <== osd.0 v2:172.21.15.37:6808/2283587593 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fea5c06a110 con 0x7fea40080f30 2024-09-17T04:33:25.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.826+0000 7f8135ffb640 1 -- 172.21.15.37:0/495517500 <== osd.7 v2:172.21.15.92:6802/3444787223 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f814009ac20 con 0x7f8124081a50 2024-09-17T04:33:25.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.826+0000 7fea63d50640 1 -- 172.21.15.37:0/3252681947 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x7fea40080f30 msgr2=0x7fea40083380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.826+0000 7fea63d50640 1 --2- 172.21.15.37:0/3252681947 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x7fea40080f30 0x7fea40083380 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.826+0000 7fea63d50640 1 -- 172.21.15.37:0/3252681947 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fea40078c60 msgr2=0x7fea4007b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.826+0000 7fea63d50640 1 --2- 172.21.15.37:0/3252681947 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fea40078c60 0x7fea4007b170 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fea5c06f110 tx=0x7fea58009260 comp rx=0 tx=0).stop 2024-09-17T04:33:25.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.826+0000 7fea63d50640 1 -- 172.21.15.37:0/3252681947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea5c070320 msgr2=0x7fea5c1f9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.826+0000 7fea63d50640 1 --2- 172.21.15.37:0/3252681947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea5c070320 0x7fea5c1f9a30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fea5400caa0 tx=0x7fea5400cea0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.827+0000 7fea622ed640 1 -- 172.21.15.37:0/3252681947 reap_dead start 2024-09-17T04:33:25.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.827+0000 7fea63d50640 1 -- 172.21.15.37:0/3252681947 shutdown_connections 2024-09-17T04:33:25.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.827+0000 7fea63d50640 1 -- 172.21.15.37:0/3252681947 >> 172.21.15.37:0/3252681947 conn(0x7fea5c0d00b0 msgr2=0x7fea5c0cac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:25.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.827+0000 7f814767e640 1 -- 172.21.15.37:0/495517500 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x7f8124081a50 msgr2=0x7f8124083f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.827+0000 7f814767e640 1 --2- 172.21.15.37:0/495517500 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x7f8124081a50 0x7f8124083f40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.827+0000 7fea63d50640 1 -- 172.21.15.37:0/3252681947 shutdown_connections 2024-09-17T04:33:25.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.827+0000 7fea63d50640 1 -- 172.21.15.37:0/3252681947 wait complete. 2024-09-17T04:33:25.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.827+0000 7f814767e640 1 -- 172.21.15.37:0/495517500 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8124078be0 msgr2=0x7f812407b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.827+0000 7f814767e640 1 --2- 172.21.15.37:0/495517500 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8124078be0 0x7f812407b0f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f813c007fd0 tx=0x7f813c009040 comp rx=0 tx=0).stop 2024-09-17T04:33:25.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.827+0000 7f814767e640 1 -- 172.21.15.37:0/495517500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81400b53e0 msgr2=0x7f81400ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:25.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.827+0000 7f814767e640 1 --2- 172.21.15.37:0/495517500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81400b53e0 0x7f81400ad130 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f813000b3e0 tx=0x7f813000b8a0 comp rx=0 tx=0).stop 2024-09-17T04:33:25.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.827+0000 7f8145c1b640 1 -- 172.21.15.37:0/495517500 reap_dead start 2024-09-17T04:33:25.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.828+0000 7f814767e640 1 -- 172.21.15.37:0/495517500 shutdown_connections 2024-09-17T04:33:25.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.828+0000 7f814767e640 1 -- 172.21.15.37:0/495517500 >> 172.21.15.37:0/495517500 conn(0x7f81400b1ca0 msgr2=0x7f8140153240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:25.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.828+0000 7f814767e640 1 -- 172.21.15.37:0/495517500 shutdown_connections 2024-09-17T04:33:25.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:25.828+0000 7f814767e640 1 -- 172.21.15.37:0/495517500 wait complete. 2024-09-17T04:33:25.856 INFO:teuthology.orchestra.run.smithi037.stdout:60129542150 2024-09-17T04:33:25.857 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T04:33:25.858 INFO:teuthology.orchestra.run.smithi037.stdout:60129542152 2024-09-17T04:33:25.858 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T04:33:25.859 INFO:teuthology.orchestra.run.smithi037.stdout:60129542151 2024-09-17T04:33:25.859 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T04:33:25.860 INFO:teuthology.orchestra.run.smithi037.stdout:60129542151 2024-09-17T04:33:25.860 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T04:33:25.861 INFO:teuthology.orchestra.run.smithi037.stdout:60129542152 2024-09-17T04:33:25.861 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T04:33:25.862 INFO:teuthology.orchestra.run.smithi037.stdout:60129542151 2024-09-17T04:33:25.862 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T04:33:25.863 INFO:teuthology.orchestra.run.smithi037.stdout:60129542151 2024-09-17T04:33:25.863 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T04:33:25.864 INFO:teuthology.orchestra.run.smithi037.stdout:60129542151 2024-09-17T04:33:25.864 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T04:33:26.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.125+0000 7fb2ce7fc640 1 -- 172.21.15.37:0/3818274678 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c002cd80 con 0x7fb2d00cdfd0 2024-09-17T04:33:26.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.126+0000 7fb2d6151640 1 -- 172.21.15.37:0/3818274678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2d00cdfd0 msgr2=0x7fb2d00ce3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.126+0000 7fb2d6151640 1 --2- 172.21.15.37:0/3818274678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2d00cdfd0 0x7fb2d00ce3e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000b330 tx=0x7fb2c002ac10 comp rx=0 tx=0).stop 2024-09-17T04:33:26.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.126+0000 7fb2d6151640 1 -- 172.21.15.37:0/3818274678 shutdown_connections 2024-09-17T04:33:26.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.126+0000 7fb2d6151640 1 --2- 172.21.15.37:0/3818274678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2d00cdfd0 0x7fb2d00ce3e0 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000b330 tx=0x7fb2c002ac10 comp rx=0 tx=0).stop 2024-09-17T04:33:26.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.126+0000 7fb2d6151640 1 -- 172.21.15.37:0/3818274678 >> 172.21.15.37:0/3818274678 conn(0x7fb2d00b1ca0 msgr2=0x7fb2d00b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:26.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.126+0000 7f13267fc640 1 -- 172.21.15.37:0/717392911 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c031bf0 con 0x7f1320057560 2024-09-17T04:33:26.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.126+0000 7f9fe3919640 1 -- 172.21.15.37:0/2189255368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fdc0d0560 msgr2=0x7f9fdc15eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.126+0000 7f9fe3919640 1 --2- 172.21.15.37:0/2189255368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fdc0d0560 0x7f9fdc15eba0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009830 tx=0x7f9fd802a5d0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.127+0000 7f132dcb2640 1 -- 172.21.15.37:0/717392911 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1320057560 msgr2=0x7f13200e3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.127+0000 7f132dcb2640 1 --2- 172.21.15.37:0/717392911 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1320057560 0x7f13200e3650 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f131c009830 tx=0x7f131c02a520 comp rx=0 tx=0).stop 2024-09-17T04:33:26.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.127+0000 7f132dcb2640 1 -- 172.21.15.37:0/717392911 shutdown_connections 2024-09-17T04:33:26.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.127+0000 7f132dcb2640 1 --2- 172.21.15.37:0/717392911 >> v2:172.21.15.37:3300/0 conn(0x7f1320053fd0 0x7f1320037490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.127+0000 7f132dcb2640 1 --2- 172.21.15.37:0/717392911 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1320057560 0x7f13200e3650 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.127+0000 7f132dcb2640 1 -- 172.21.15.37:0/717392911 >> 172.21.15.37:0/717392911 conn(0x7f1320055e70 msgr2=0x7f132000e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:26.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.127+0000 7fb2d6151640 1 -- 172.21.15.37:0/3818274678 shutdown_connections 2024-09-17T04:33:26.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.127+0000 7f9fe3919640 1 -- 172.21.15.37:0/2189255368 shutdown_connections 2024-09-17T04:33:26.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.127+0000 7f9fe3919640 1 --2- 172.21.15.37:0/2189255368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fdc0d0560 0x7f9fdc15eba0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.127+0000 7f9fe3919640 1 -- 172.21.15.37:0/2189255368 >> 172.21.15.37:0/2189255368 conn(0x7f9fdc0b2bb0 msgr2=0x7f9fdc0b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:26.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.127+0000 7fb2d6151640 1 -- 172.21.15.37:0/3818274678 wait complete. 2024-09-17T04:33:26.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.127+0000 7f132dcb2640 1 -- 172.21.15.37:0/717392911 shutdown_connections 2024-09-17T04:33:26.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.127+0000 7f9fe3919640 1 -- 172.21.15.37:0/2189255368 shutdown_connections 2024-09-17T04:33:26.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.127+0000 7f132dcb2640 1 -- 172.21.15.37:0/717392911 wait complete. 2024-09-17T04:33:26.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.128+0000 7fac50157640 1 -- 172.21.15.37:0/1072175183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac480b0de0 msgr2=0x7fac480b11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.128+0000 7fac50157640 1 --2- 172.21.15.37:0/1072175183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac480b0de0 0x7fac480b11f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fac40005fd0 tx=0x7fac4002a980 comp rx=0 tx=0).stop 2024-09-17T04:33:26.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.128+0000 7fac50157640 1 -- 172.21.15.37:0/1072175183 shutdown_connections 2024-09-17T04:33:26.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.128+0000 7fac50157640 1 --2- 172.21.15.37:0/1072175183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac480b0de0 0x7fac480b11f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.128+0000 7fac50157640 1 --2- 172.21.15.37:0/1072175183 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fac480b6640 0x7fac480b0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.128+0000 7fac50157640 1 -- 172.21.15.37:0/1072175183 >> 172.21.15.37:0/1072175183 conn(0x7fac4809aa20 msgr2=0x7fac480b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:26.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.128+0000 7fac50157640 1 -- 172.21.15.37:0/1072175183 shutdown_connections 2024-09-17T04:33:26.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.129+0000 7fb604c32640 1 -- 172.21.15.37:0/2456635889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5f80df390 msgr2=0x7fb5f8007680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.129+0000 7fb604c32640 1 --2- 172.21.15.37:0/2456635889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5f80df390 0x7fb5f8007680 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4009820 tx=0x7fb5f402a510 comp rx=0 tx=0).stop 2024-09-17T04:33:26.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.130+0000 7fb604c32640 1 -- 172.21.15.37:0/2456635889 shutdown_connections 2024-09-17T04:33:26.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.130+0000 7fb604c32640 1 --2- 172.21.15.37:0/2456635889 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb5f8004a70 0x7fb5f80060a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.130+0000 7fb604c32640 1 --2- 172.21.15.37:0/2456635889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5f80df390 0x7fb5f8007680 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.130+0000 7fb604c32640 1 -- 172.21.15.37:0/2456635889 >> 172.21.15.37:0/2456635889 conn(0x7fb5f801ddf0 msgr2=0x7fb5f8039ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:26.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.130+0000 7fe4ad22e640 1 -- 172.21.15.37:0/4180674479 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a00047a0 con 0x7fe4a8075b90 2024-09-17T04:33:26.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.130+0000 7fb604c32640 1 -- 172.21.15.37:0/2456635889 shutdown_connections 2024-09-17T04:33:26.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.130+0000 7fb604c32640 1 -- 172.21.15.37:0/2456635889 wait complete. 2024-09-17T04:33:26.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.130+0000 7f132dcb2640 1 Processor -- start 2024-09-17T04:33:26.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.127+0000 7f9fe3919640 1 -- 172.21.15.37:0/2189255368 wait complete. 2024-09-17T04:33:26.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.130+0000 7f9fe3919640 1 Processor -- start 2024-09-17T04:33:26.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.130+0000 7f9fe3919640 1 -- start start 2024-09-17T04:33:26.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.130+0000 7f132dcb2640 1 -- start start 2024-09-17T04:33:26.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.130+0000 7fb2d6151640 1 Processor -- start 2024-09-17T04:33:26.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.130+0000 7fb2d6151640 1 -- start start 2024-09-17T04:33:26.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.128+0000 7fac50157640 1 -- 172.21.15.37:0/1072175183 wait complete. 2024-09-17T04:33:26.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.131+0000 7fe4b0494640 1 -- 172.21.15.37:0/4180674479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4a8075b90 msgr2=0x7fe4a8154bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.131+0000 7fe4b0494640 1 --2- 172.21.15.37:0/4180674479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4a8075b90 0x7fe4a8154bb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8072c60 tx=0x7fe4a002a560 comp rx=0 tx=0).stop 2024-09-17T04:33:26.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.131+0000 7fac50157640 1 Processor -- start 2024-09-17T04:33:26.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.131+0000 7efef759e640 1 -- 172.21.15.37:0/1883849716 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7efef0036b10 msgr2=0x7efef00df180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.131+0000 7efef759e640 1 --2- 172.21.15.37:0/1883849716 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7efef0036b10 0x7efef00df180 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efef8066ac0 tx=0x7efef80a63d0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.131+0000 7fe4b0494640 1 -- 172.21.15.37:0/4180674479 shutdown_connections 2024-09-17T04:33:26.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.132+0000 7fe4b0494640 1 --2- 172.21.15.37:0/4180674479 >> v2:172.21.15.37:3300/0 conn(0x7fe4a8073f70 0x7fe4a80c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.132+0000 7fe4b0494640 1 --2- 172.21.15.37:0/4180674479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4a8075b90 0x7fe4a8154bb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.132+0000 7fe4b0494640 1 -- 172.21.15.37:0/4180674479 >> 172.21.15.37:0/4180674479 conn(0x7fe4a80b20e0 msgr2=0x7fe4a80b24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:26.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.132+0000 7efef759e640 1 -- 172.21.15.37:0/1883849716 shutdown_connections 2024-09-17T04:33:26.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.132+0000 7efef759e640 1 --2- 172.21.15.37:0/1883849716 >> v2:172.21.15.92:3300/0 conn(0x7efef0009ba0 0x7efef0005cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.132+0000 7efef759e640 1 --2- 172.21.15.37:0/1883849716 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7efef0036b10 0x7efef00df180 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.132+0000 7efef759e640 1 -- 172.21.15.37:0/1883849716 >> 172.21.15.37:0/1883849716 conn(0x7efef0051420 msgr2=0x7efef0051860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:26.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.131+0000 7fac50157640 1 -- start start 2024-09-17T04:33:26.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.132+0000 7fe4b0494640 1 -- 172.21.15.37:0/4180674479 shutdown_connections 2024-09-17T04:33:26.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.132+0000 7fe4b0494640 1 -- 172.21.15.37:0/4180674479 wait complete. 2024-09-17T04:33:26.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.132+0000 7ff7f7447640 1 -- 172.21.15.37:0/2120136210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7e8036400 msgr2=0x7ff7e8036810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.132+0000 7ff7f7447640 1 --2- 172.21.15.37:0/2120136210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7e8036400 0x7ff7e8036810 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff7f00774d0 tx=0x7ff7f00a6470 comp rx=0 tx=0).stop 2024-09-17T04:33:26.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.132+0000 7ff7f7447640 1 -- 172.21.15.37:0/2120136210 shutdown_connections 2024-09-17T04:33:26.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7ff7f7447640 1 --2- 172.21.15.37:0/2120136210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7e8036400 0x7ff7e8036810 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7ff7f7447640 1 --2- 172.21.15.37:0/2120136210 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff7e800bc30 0x7ff7e8005cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7fb2d6151640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2d006f480 0x7fb2d0074360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.132+0000 7efef759e640 1 -- 172.21.15.37:0/1883849716 shutdown_connections 2024-09-17T04:33:26.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7ff7f7447640 1 -- 172.21.15.37:0/2120136210 >> 172.21.15.37:0/2120136210 conn(0x7ff7e8051420 msgr2=0x7ff7e8051860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:26.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7efef759e640 1 -- 172.21.15.37:0/1883849716 wait complete. 2024-09-17T04:33:26.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7fb2d6151640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb2d00ccae0 0x7fb2d01f49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7fb2d6151640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2d0151160 0x7fb2d01f4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7ff7f7447640 1 -- 172.21.15.37:0/2120136210 shutdown_connections 2024-09-17T04:33:26.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7fb604c32640 1 Processor -- start 2024-09-17T04:33:26.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7f9fe3919640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9fdc070070 0x7f9fdc0aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7ff7f7447640 1 -- 172.21.15.37:0/2120136210 wait complete. 2024-09-17T04:33:26.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7f132dcb2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1320057560 0x7f13200d9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7f132dcb2640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f13200e0c10 0x7f13200dbb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7f132dcb2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13200dc0b0 0x7f13200dc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7f132dcb2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1320037ea0 con 0x7f1320057560 2024-09-17T04:33:26.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7f1326ffd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f13200e0c10 0x7f13200dbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7f1327fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13200dc0b0 0x7f13200dc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.134+0000 7f9fe3919640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fdc0d0560 0x7f9fdc2051a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7f13277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1320057560 0x7f13200d9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7f1327fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13200dc0b0 0x7f13200dc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.37:50734/0 (socket says 172.21.15.37:50734) 2024-09-17T04:33:26.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.134+0000 7f9fe16b5640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9fdc070070 0x7f9fdc0aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.134+0000 7f9fe16b5640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9fdc070070 0x7f9fdc0aeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.37:38762/0 (socket says 172.21.15.37:38762) 2024-09-17T04:33:26.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.134+0000 7f1327fff640 1 -- 172.21.15.37:0/1838546434 learned_addr learned my addr 172.21.15.37:0/1838546434 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:26.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.134+0000 7f9fe16b5640 1 -- 172.21.15.37:0/2467013784 learned_addr learned my addr 172.21.15.37:0/2467013784 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:26.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.134+0000 7fac50157640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fac480b0de0 0x7fac48155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.134+0000 7fac50157640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac480b6640 0x7fac4814df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.134+0000 7fac50157640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac480cda50 0x7fac481fd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.134+0000 7fac50157640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fac480b1cd0 con 0x7fac480cda50 2024-09-17T04:33:26.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.134+0000 7fac50157640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fac480b1b50 con 0x7fac480b6640 2024-09-17T04:33:26.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.134+0000 7fac50157640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fac480b1e50 con 0x7fac480b0de0 2024-09-17T04:33:26.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.134+0000 7f132dcb2640 1 -- 172.21.15.37:0/1838546434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1320037d20 con 0x7f13200dc0b0 2024-09-17T04:33:26.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.134+0000 7f9fe0eb4640 1 --2- 172.21.15.37:0/2467013784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fdc0d0560 0x7f9fdc2051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.134+0000 7f9fe3919640 1 --2- 172.21.15.37:0/2467013784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fdc0af030 0x7f9fdc2056e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.134+0000 7f132dcb2640 1 -- 172.21.15.37:0/1838546434 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f1320038020 con 0x7f13200e0c10 2024-09-17T04:33:26.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.134+0000 7fac4d6f2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac480b6640 0x7fac4814df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7fb2cf7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2d006f480 0x7fb2d0074360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7fb2cf7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2d006f480 0x7fb2d0074360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.37:50718/0 (socket says 172.21.15.37:50718) 2024-09-17T04:33:26.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7fb2ceffd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb2d00ccae0 0x7fb2d01f49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7fb2ceffd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb2d00ccae0 0x7fb2d01f49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.37:38758/0 (socket says 172.21.15.37:38758) 2024-09-17T04:33:26.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7fb2cffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2d0151160 0x7fb2d01f4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7fb2d6151640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d00af120 con 0x7fb2d0151160 2024-09-17T04:33:26.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7fb2d6151640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d00aefa0 con 0x7fb2d006f480 2024-09-17T04:33:26.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7fb2d6151640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fb2d00af2a0 con 0x7fb2d00ccae0 2024-09-17T04:33:26.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7fb2cffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2d0151160 0x7fb2d01f4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45728/0 (socket says 172.21.15.37:45728) 2024-09-17T04:33:26.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.134+0000 7fb2cffff640 1 -- 172.21.15.37:0/4053142231 learned_addr learned my addr 172.21.15.37:0/4053142231 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:26.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.134+0000 7fac4d6f2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac480b6640 0x7fac4814df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.37:50754/0 (socket says 172.21.15.37:50754) 2024-09-17T04:33:26.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.134+0000 7fac4d6f2640 1 -- 172.21.15.37:0/1717526960 learned_addr learned my addr 172.21.15.37:0/1717526960 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:26.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.134+0000 7fb2cffff640 1 -- 172.21.15.37:0/4053142231 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb2d00ccae0 msgr2=0x7fb2d01f49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.134+0000 7fac4e6f4640 1 --2- 172.21.15.37:0/1717526960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac480cda50 0x7fac481fd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.134+0000 7fb2cffff640 1 --2- 172.21.15.37:0/4053142231 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb2d00ccae0 0x7fb2d01f49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.134+0000 7fb2cffff640 1 -- 172.21.15.37:0/4053142231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2d006f480 msgr2=0x7fb2d0074360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.134+0000 7fac4def3640 1 --2- 172.21.15.37:0/1717526960 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fac480b0de0 0x7fac48155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.135+0000 7f1327fff640 1 -- 172.21.15.37:0/1838546434 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f13200e0c10 msgr2=0x7f13200dbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.135+0000 7fac4e6f4640 1 -- 172.21.15.37:0/1717526960 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fac480b0de0 msgr2=0x7fac48155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.135+0000 7fac4e6f4640 1 --2- 172.21.15.37:0/1717526960 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fac480b0de0 0x7fac48155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.135+0000 7fac4e6f4640 1 -- 172.21.15.37:0/1717526960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac480b6640 msgr2=0x7fac4814df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.135+0000 7fac4e6f4640 1 --2- 172.21.15.37:0/1717526960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac480b6640 0x7fac4814df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.135+0000 7fac4e6f4640 1 -- 172.21.15.37:0/1717526960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac481565e0 con 0x7fac480cda50 2024-09-17T04:33:26.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.135+0000 7f1327fff640 1 --2- 172.21.15.37:0/1838546434 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f13200e0c10 0x7f13200dbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.135+0000 7f9fe3919640 1 -- 172.21.15.37:0/2467013784 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9fdc160e00 con 0x7f9fdc0af030 2024-09-17T04:33:26.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.135+0000 7f9fe3919640 1 -- 172.21.15.37:0/2467013784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9fdc160c80 con 0x7f9fdc0d0560 2024-09-17T04:33:26.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.135+0000 7f9fe3919640 1 -- 172.21.15.37:0/2467013784 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f9fdc160f80 con 0x7f9fdc070070 2024-09-17T04:33:26.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.135+0000 7f1327fff640 1 -- 172.21.15.37:0/1838546434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1320057560 msgr2=0x7f13200d9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.135+0000 7f9fe1eb6640 1 --2- 172.21.15.37:0/2467013784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fdc0af030 0x7f9fdc2056e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.135+0000 7f1327fff640 1 --2- 172.21.15.37:0/1838546434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1320057560 0x7f13200d9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.135+0000 7f1327fff640 1 -- 172.21.15.37:0/1838546434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13200dae30 con 0x7f13200dc0b0 2024-09-17T04:33:26.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.135+0000 7f13277fe640 1 --2- 172.21.15.37:0/1838546434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1320057560 0x7f13200d9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:33:26.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.134+0000 7fb2cffff640 1 --2- 172.21.15.37:0/4053142231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2d006f480 0x7fb2d0074360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.134+0000 7fb2cffff640 1 -- 172.21.15.37:0/4053142231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d0074980 con 0x7fb2d0151160 2024-09-17T04:33:26.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.135+0000 7fac4e6f4640 1 --2- 172.21.15.37:0/1717526960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac480cda50 0x7fac481fd3c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fac40032000 tx=0x7fac40003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:26.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.135+0000 7f1327fff640 1 --2- 172.21.15.37:0/1838546434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13200dc0b0 0x7f13200dc4c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1328066830 tx=0x7f132806a0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:26.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.133+0000 7fb604c32640 1 -- start start 2024-09-17T04:33:26.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.135+0000 7fb604c32640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5f8004a70 0x7fb5f80ed8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.135+0000 7fb604c32640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb5f80065e0 0x7fb5f80dd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7f9fe1eb6640 1 -- 172.21.15.37:0/2467013784 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9fdc070070 msgr2=0x7f9fdc0aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7fe4b0494640 1 Processor -- start 2024-09-17T04:33:26.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7fe4b0494640 1 -- start start 2024-09-17T04:33:26.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7fb5ff7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5f8004a70 0x7fb5f80ed8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7fb5ff7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5f8004a70 0x7fb5f80ed8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.37:50768/0 (socket says 172.21.15.37:50768) 2024-09-17T04:33:26.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7fb5feffd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb5f80065e0 0x7fb5f80dd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7fb5feffd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb5f80065e0 0x7fb5f80dd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.37:38770/0 (socket says 172.21.15.37:38770) 2024-09-17T04:33:26.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7fb604c32640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5f80df390 0x7fb5f80dd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7f9fe1eb6640 1 --2- 172.21.15.37:0/2467013784 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9fdc070070 0x7f9fdc0aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7fb604c32640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f803a340 con 0x7fb5f80df390 2024-09-17T04:33:26.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7fb604c32640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f803a1c0 con 0x7fb5f8004a70 2024-09-17T04:33:26.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7fb604c32640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fb5f8055280 con 0x7fb5f80065e0 2024-09-17T04:33:26.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7fb5ff7fe640 1 -- 172.21.15.37:0/717349158 learned_addr learned my addr 172.21.15.37:0/717349158 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:26.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7fb5fffff640 1 --2- 172.21.15.37:0/717349158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5f80df390 0x7fb5f80dd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7f9fe1eb6640 1 -- 172.21.15.37:0/2467013784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fdc0d0560 msgr2=0x7f9fdc2051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7f9fe1eb6640 1 --2- 172.21.15.37:0/2467013784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fdc0d0560 0x7f9fdc2051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7f9fe1eb6640 1 -- 172.21.15.37:0/2467013784 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fdc0ad4f0 con 0x7f9fdc0af030 2024-09-17T04:33:26.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7f9fe0eb4640 1 --2- 172.21.15.37:0/2467013784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fdc0d0560 0x7f9fdc2051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:33:26.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7f9fe1eb6640 1 --2- 172.21.15.37:0/2467013784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fdc0af030 0x7f9fdc2056e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9fd400c730 tx=0x7f9fd400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:26.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7fac3e7fc640 1 -- 172.21.15.37:0/1717526960 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac40034040 con 0x7fac480cda50 2024-09-17T04:33:26.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7fac3e7fc640 1 -- 172.21.15.37:0/1717526960 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac40004540 con 0x7fac480cda50 2024-09-17T04:33:26.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7fac3e7fc640 1 -- 172.21.15.37:0/1717526960 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac40045480 con 0x7fac480cda50 2024-09-17T04:33:26.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7fb2cffff640 1 --2- 172.21.15.37:0/4053142231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2d0151160 0x7fb2d01f4ef0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8014300 tx=0x7fb2c8014700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:26.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7fac50157640 1 -- 172.21.15.37:0/1717526960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac481568c0 con 0x7fac480cda50 2024-09-17T04:33:26.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7fac50157640 1 -- 172.21.15.37:0/1717526960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac481fb100 con 0x7fac480cda50 2024-09-17T04:33:26.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.137+0000 7fb5feffd640 1 -- 172.21.15.37:0/717349158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5f8004a70 msgr2=0x7fb5f80ed8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.137+0000 7fb5feffd640 1 --2- 172.21.15.37:0/717349158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5f8004a70 0x7fb5f80ed8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.137+0000 7fb5feffd640 1 -- 172.21.15.37:0/717349158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5f80df390 msgr2=0x7fb5f80dd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.137+0000 7fb5feffd640 1 --2- 172.21.15.37:0/717349158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5f80df390 0x7fb5f80dd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.137+0000 7fb5feffd640 1 -- 172.21.15.37:0/717349158 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f80decd0 con 0x7fb5f80065e0 2024-09-17T04:33:26.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.137+0000 7efef759e640 1 Processor -- start 2024-09-17T04:33:26.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.137+0000 7ff7f7447640 1 Processor -- start 2024-09-17T04:33:26.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.137+0000 7f9fd1ffb640 1 -- 172.21.15.37:0/2467013784 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4003b70 con 0x7f9fdc0af030 2024-09-17T04:33:26.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.137+0000 7f9fd1ffb640 1 -- 172.21.15.37:0/2467013784 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fd4003d10 con 0x7f9fdc0af030 2024-09-17T04:33:26.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.137+0000 7f9fd1ffb640 1 -- 172.21.15.37:0/2467013784 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4005330 con 0x7f9fdc0af030 2024-09-17T04:33:26.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.137+0000 7efef759e640 1 -- start start 2024-09-17T04:33:26.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.137+0000 7ff7f7447640 1 -- start start 2024-09-17T04:33:26.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.137+0000 7fb5fffff640 1 --2- 172.21.15.37:0/717349158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5f80df390 0x7fb5f80dd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:33:26.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.137+0000 7f9fe3919640 1 -- 172.21.15.37:0/2467013784 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fdc157420 con 0x7f9fdc0af030 2024-09-17T04:33:26.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.137+0000 7fb5feffd640 1 --2- 172.21.15.37:0/717349158 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb5f80065e0 0x7fb5f80dd420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00e6e0 tx=0x7fb5ec00eba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:26.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.137+0000 7f9fe3919640 1 -- 172.21.15.37:0/2467013784 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fdc1579e0 con 0x7f9fdc0af030 2024-09-17T04:33:26.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.138+0000 7fe4b0494640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe4a8073f70 0x7fe4a80b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.138+0000 7fe4b0494640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4a8075b90 0x7fe4a80c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.138+0000 7fe4ae230640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe4a8073f70 0x7fe4a80b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.138+0000 7fe4ae230640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe4a8073f70 0x7fe4a80b4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.37:38778/0 (socket says 172.21.15.37:38778) 2024-09-17T04:33:26.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.138+0000 7fe4ae230640 1 -- 172.21.15.37:0/2975744756 learned_addr learned my addr 172.21.15.37:0/2975744756 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:26.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7f1313fff640 1 -- 172.21.15.37:0/1838546434 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1328098070 con 0x7f13200dc0b0 2024-09-17T04:33:26.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7f1313fff640 1 -- 172.21.15.37:0/1838546434 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1328072d70 con 0x7f13200dc0b0 2024-09-17T04:33:26.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.136+0000 7f1313fff640 1 -- 172.21.15.37:0/1838546434 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1328093670 con 0x7f13200dc0b0 2024-09-17T04:33:26.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.137+0000 7f132dcb2640 1 -- 172.21.15.37:0/1838546434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13200db0d0 con 0x7f13200dc0b0 2024-09-17T04:33:26.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.137+0000 7f132dcb2640 1 -- 172.21.15.37:0/1838546434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f132016bb10 con 0x7f13200dc0b0 2024-09-17T04:33:26.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.139+0000 7f132dcb2640 1 -- 172.21.15.37:0/1838546434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1320037490 con 0x7f13200dc0b0 2024-09-17T04:33:26.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.139+0000 7fb5ebfff640 1 -- 172.21.15.37:0/717349158 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec00cab0 con 0x7fb5f80065e0 2024-09-17T04:33:26.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.139+0000 7fb604c32640 1 -- 172.21.15.37:0/717349158 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f8050bf0 con 0x7fb5f80065e0 2024-09-17T04:33:26.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.139+0000 7fb604c32640 1 -- 172.21.15.37:0/717349158 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f80de010 con 0x7fb5f80065e0 2024-09-17T04:33:26.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.139+0000 7fb5ebfff640 1 -- 172.21.15.37:0/717349158 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5ec00cc50 con 0x7fb5f80065e0 2024-09-17T04:33:26.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.139+0000 7fe4b0494640 1 --2- 172.21.15.37:0/2975744756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a80c5da0 0x7fe4a80c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.139+0000 7fe4b0494640 1 -- 172.21.15.37:0/2975744756 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a80ce9f0 con 0x7fe4a80c5da0 2024-09-17T04:33:26.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.140+0000 7fac50157640 1 -- 172.21.15.37:0/1717526960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac480b11f0 con 0x7fac480cda50 2024-09-17T04:33:26.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.140+0000 7efef759e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efef0009ba0 0x7efef00de8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.140+0000 7ff7f7447640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7e8009ba0 0x7ff7e80defa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.139+0000 7fb2b7fff640 1 -- 172.21.15.37:0/4053142231 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8003970 con 0x7fb2d0151160 2024-09-17T04:33:26.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.140+0000 7fb2b7fff640 1 -- 172.21.15.37:0/4053142231 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2c8003b30 con 0x7fb2d0151160 2024-09-17T04:33:26.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.140+0000 7efef759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efef000bc30 0x7efef017db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.140+0000 7fe4b0494640 1 -- 172.21.15.37:0/2975744756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a80ce870 con 0x7fe4a8075b90 2024-09-17T04:33:26.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.140+0000 7fe4b0494640 1 -- 172.21.15.37:0/2975744756 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fe4a80ceb70 con 0x7fe4a8073f70 2024-09-17T04:33:26.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.140+0000 7efef759e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7efef0036b10 0x7efef017e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.140+0000 7ff7f7447640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff7e800bc30 0x7ff7e8020460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.140+0000 7f9fbf7fe640 1 -- 172.21.15.37:0/2467013784 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fdc15eba0 con 0x7f9fdc0af030 2024-09-17T04:33:26.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.140+0000 7efef759e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efef0006ac0 con 0x7efef000bc30 2024-09-17T04:33:26.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.140+0000 7fb5ebfff640 1 -- 172.21.15.37:0/717349158 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec010690 con 0x7fb5f80065e0 2024-09-17T04:33:26.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.140+0000 7efef759e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efef0006940 con 0x7efef0009ba0 2024-09-17T04:33:26.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.140+0000 7efef759e640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7efef0006c40 con 0x7efef0036b10 2024-09-17T04:33:26.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.140+0000 7ff7f6445640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7e8009ba0 0x7ff7e80defa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.140+0000 7ff7f6445640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7e8009ba0 0x7ff7e80defa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.37:50794/0 (socket says 172.21.15.37:50794) 2024-09-17T04:33:26.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.140+0000 7efef659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efef0009ba0 0x7efef00de8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.140+0000 7ff7f7447640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7e8036400 0x7ff7e80209a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.140+0000 7ff7f7447640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e8006a40 con 0x7ff7e8036400 2024-09-17T04:33:26.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.140+0000 7ff7f7447640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e80068c0 con 0x7ff7e8009ba0 2024-09-17T04:33:26.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.140+0000 7ff7f7447640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7ff7e8006bc0 con 0x7ff7e800bc30 2024-09-17T04:33:26.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.140+0000 7efef6d9d640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7efef0036b10 0x7efef017e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.140+0000 7efef659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efef0009ba0 0x7efef00de8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.37:50788/0 (socket says 172.21.15.37:50788) 2024-09-17T04:33:26.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.140+0000 7ff7f6445640 1 -- 172.21.15.37:0/3323656362 learned_addr learned my addr 172.21.15.37:0/3323656362 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:26.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.140+0000 7ff7f5c44640 1 --2- 172.21.15.37:0/3323656362 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff7e800bc30 0x7ff7e8020460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.140+0000 7f9fd1ffb640 1 -- 172.21.15.37:0/2467013784 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9fd40056e0 con 0x7f9fdc0af030 2024-09-17T04:33:26.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.140+0000 7efef659c640 1 -- 172.21.15.37:0/2295979183 learned_addr learned my addr 172.21.15.37:0/2295979183 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:26.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.141+0000 7fe4ada2f640 1 --2- 172.21.15.37:0/2975744756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4a8075b90 0x7fe4a80c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.141+0000 7efef659c640 1 -- 172.21.15.37:0/2295979183 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7efef0036b10 msgr2=0x7efef017e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.141+0000 7efef659c640 1 --2- 172.21.15.37:0/2295979183 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7efef0036b10 0x7efef017e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.141+0000 7efef659c640 1 -- 172.21.15.37:0/2295979183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efef000bc30 msgr2=0x7efef017db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.141+0000 7efef659c640 1 --2- 172.21.15.37:0/2295979183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efef000bc30 0x7efef017db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.141+0000 7fb2b7fff640 1 -- 172.21.15.37:0/4053142231 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c80120d0 con 0x7fb2d0151160 2024-09-17T04:33:26.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.141+0000 7fb2d6151640 1 -- 172.21.15.37:0/4053142231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d01e3010 con 0x7fb2d0151160 2024-09-17T04:33:26.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.141+0000 7efef659c640 1 -- 172.21.15.37:0/2295979183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef002ccf0 con 0x7efef0009ba0 2024-09-17T04:33:26.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.141+0000 7efef6d9d640 1 --2- 172.21.15.37:0/2295979183 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7efef0036b10 0x7efef017e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:33:26.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.141+0000 7ff7f6c46640 1 --2- 172.21.15.37:0/3323656362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7e8036400 0x7ff7e80209a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.141+0000 7fe4ae230640 1 -- 172.21.15.37:0/2975744756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4a8075b90 msgr2=0x7fe4a80c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.141+0000 7fe4ae230640 1 --2- 172.21.15.37:0/2975744756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4a8075b90 0x7fe4a80c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.141+0000 7fe4ae230640 1 -- 172.21.15.37:0/2975744756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a80c5da0 msgr2=0x7fe4a80c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.141+0000 7fe4ae230640 1 --2- 172.21.15.37:0/2975744756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a80c5da0 0x7fe4a80c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.141+0000 7fe4ae230640 1 -- 172.21.15.37:0/2975744756 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a80b6210 con 0x7fe4a8073f70 2024-09-17T04:33:26.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.141+0000 7f9fd1ffb640 1 --2- 172.21.15.37:0/2467013784 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9fc0078c90 0x7f9fc007b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.141+0000 7fb2d6151640 1 -- 172.21.15.37:0/4053142231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d01413e0 con 0x7fb2d0151160 2024-09-17T04:33:26.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.141+0000 7f9fd1ffb640 1 -- 172.21.15.37:0/2467013784 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9fd409ab80 con 0x7f9fdc0af030 2024-09-17T04:33:26.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.141+0000 7fe4ae230640 1 --2- 172.21.15.37:0/2975744756 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe4a8073f70 0x7fe4a80b4290 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0034040 tx=0x7fe4a0003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:26.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.141+0000 7fb604c32640 1 -- 172.21.15.37:0/717349158 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e0005290 con 0x7fb5f80065e0 2024-09-17T04:33:26.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.141+0000 7efef659c640 1 --2- 172.21.15.37:0/2295979183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efef0009ba0 0x7efef00de8e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efee800c420 tx=0x7efee800c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:26.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.142+0000 7f1313fff640 1 -- 172.21.15.37:0/1838546434 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f132809b070 con 0x7f13200dc0b0 2024-09-17T04:33:26.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.142+0000 7fe4a6ffd640 1 -- 172.21.15.37:0/2975744756 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a003e070 con 0x7fe4a8073f70 2024-09-17T04:33:26.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.142+0000 7ff7f6c46640 1 -- 172.21.15.37:0/3323656362 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff7e800bc30 msgr2=0x7ff7e8020460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.142+0000 7ff7f6c46640 1 --2- 172.21.15.37:0/3323656362 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff7e800bc30 0x7ff7e8020460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.142+0000 7ff7f6c46640 1 -- 172.21.15.37:0/3323656362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7e8009ba0 msgr2=0x7ff7e80defa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.142+0000 7ff7f6c46640 1 --2- 172.21.15.37:0/3323656362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7e8009ba0 0x7ff7e80defa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.142+0000 7ff7f6c46640 1 -- 172.21.15.37:0/3323656362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e8047f90 con 0x7ff7e8036400 2024-09-17T04:33:26.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.142+0000 7fe4a6ffd640 1 -- 172.21.15.37:0/2975744756 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4a0004a90 con 0x7fe4a8073f70 2024-09-17T04:33:26.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.142+0000 7fe4b0494640 1 -- 172.21.15.37:0/2975744756 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a8149d00 con 0x7fe4a8073f70 2024-09-17T04:33:26.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.142+0000 7ff7f6c46640 1 --2- 172.21.15.37:0/3323656362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7e8036400 0x7ff7e80209a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0052860 tx=0x7ff7f00ac5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:26.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.142+0000 7fe4b0494640 1 -- 172.21.15.37:0/2975744756 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a814a2c0 con 0x7fe4a8073f70 2024-09-17T04:33:26.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.142+0000 7fe4a6ffd640 1 -- 172.21.15.37:0/2975744756 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0035040 con 0x7fe4a8073f70 2024-09-17T04:33:26.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.142+0000 7f9fe16b5640 1 --2- 172.21.15.37:0/2467013784 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9fc0078c90 0x7f9fc007b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.142+0000 7fac3e7fc640 1 -- 172.21.15.37:0/1717526960 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fac40037020 con 0x7fac480cda50 2024-09-17T04:33:26.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.143+0000 7f1313fff640 1 --2- 172.21.15.37:0/1838546434 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f130c078c10 0x7f130c07b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.143+0000 7fb5ebfff640 1 -- 172.21.15.37:0/717349158 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb5ec010830 con 0x7fb5f80065e0 2024-09-17T04:33:26.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.143+0000 7fb2d6151640 1 -- 172.21.15.37:0/4053142231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2d0141640 con 0x7fb2d0151160 2024-09-17T04:33:26.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.143+0000 7f1313fff640 1 -- 172.21.15.37:0/1838546434 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1328117270 con 0x7f13200dc0b0 2024-09-17T04:33:26.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.143+0000 7f13277fe640 1 --2- 172.21.15.37:0/1838546434 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f130c078c10 0x7f130c07b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.143+0000 7f9fe16b5640 1 --2- 172.21.15.37:0/2467013784 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9fc0078c90 0x7f9fc007b1a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9fd80095f0 tx=0x7f9fd8004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:26.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.143+0000 7fac3e7fc640 1 --2- 172.21.15.37:0/1717526960 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fac2c078c60 0x7fac2c07b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.143+0000 7fac3e7fc640 1 -- 172.21.15.37:0/1717526960 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fac400bc3b0 con 0x7fac480cda50 2024-09-17T04:33:26.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.144+0000 7fb5ebfff640 1 --2- 172.21.15.37:0/717349158 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb5e4078c90 0x7fb5e407b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.144+0000 7f13277fe640 1 --2- 172.21.15.37:0/1838546434 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f130c078c10 0x7f130c07b120 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f131c034000 tx=0x7f131c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:26.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.144+0000 7fe4b0494640 1 -- 172.21.15.37:0/2975744756 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe488005290 con 0x7fe4a8073f70 2024-09-17T04:33:26.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.144+0000 7fac4def3640 1 --2- 172.21.15.37:0/1717526960 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fac2c078c60 0x7fac2c07b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.144+0000 7fb5ff7fe640 1 --2- 172.21.15.37:0/717349158 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb5e4078c90 0x7fb5e407b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.144+0000 7fb5ebfff640 1 -- 172.21.15.37:0/717349158 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb5ec09b360 con 0x7fb5f80065e0 2024-09-17T04:33:26.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.144+0000 7fe4a6ffd640 1 -- 172.21.15.37:0/2975744756 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe4a0041070 con 0x7fe4a8073f70 2024-09-17T04:33:26.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.144+0000 7fac4def3640 1 --2- 172.21.15.37:0/1717526960 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fac2c078c60 0x7fac2c07b170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fac38009b30 tx=0x7fac38005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:26.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.144+0000 7fb5ff7fe640 1 --2- 172.21.15.37:0/717349158 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb5e4078c90 0x7fb5e407b1a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4009590 tx=0x7fb5f4004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:26.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.145+0000 7fe4a6ffd640 1 --2- 172.21.15.37:0/2975744756 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe494078c10 0x7fe49407b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.145+0000 7fe4a6ffd640 1 -- 172.21.15.37:0/2975744756 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe4a00bb710 con 0x7fe4a8073f70 2024-09-17T04:33:26.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.145+0000 7fe4ada2f640 1 --2- 172.21.15.37:0/2975744756 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe494078c10 0x7fe49407b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.146+0000 7fe4ada2f640 1 --2- 172.21.15.37:0/2975744756 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe494078c10 0x7fe49407b120 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe4a80cce80 tx=0x7fe498009500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:26.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.148+0000 7f9fd1ffb640 1 -- 172.21.15.37:0/2467013784 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9fdc0ad4f0 con 0x7f9fdc0af030 2024-09-17T04:33:26.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.148+0000 7efee6ffd640 1 -- 172.21.15.37:0/2295979183 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8011790 con 0x7efef0009ba0 2024-09-17T04:33:26.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.148+0000 7efee6ffd640 1 -- 172.21.15.37:0/2295979183 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efee8017070 con 0x7efef0009ba0 2024-09-17T04:33:26.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.148+0000 7efef759e640 1 -- 172.21.15.37:0/2295979183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef002cfd0 con 0x7efef0009ba0 2024-09-17T04:33:26.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.148+0000 7efee6ffd640 1 -- 172.21.15.37:0/2295979183 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8011cc0 con 0x7efef0009ba0 2024-09-17T04:33:26.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.148+0000 7f1313fff640 1 -- 172.21.15.37:0/1838546434 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f13200dae30 con 0x7f13200dc0b0 2024-09-17T04:33:26.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.148+0000 7efef759e640 1 -- 172.21.15.37:0/2295979183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef001fc50 con 0x7efef0009ba0 2024-09-17T04:33:26.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.150+0000 7efee4ff9640 1 -- 172.21.15.37:0/2295979183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efed0005290 con 0x7efef0009ba0 2024-09-17T04:33:26.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.150+0000 7fb2b7fff640 1 -- 172.21.15.37:0/4053142231 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb2c8012270 con 0x7fb2d0151160 2024-09-17T04:33:26.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.151+0000 7fb2b7fff640 1 --2- 172.21.15.37:0/4053142231 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb2bc078be0 0x7fb2bc07b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.151+0000 7fb2cf7fe640 1 --2- 172.21.15.37:0/4053142231 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb2bc078be0 0x7fb2bc07b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.151+0000 7efee6ffd640 1 -- 172.21.15.37:0/2295979183 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efee8036d10 con 0x7efef0009ba0 2024-09-17T04:33:26.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.151+0000 7fb5ebfff640 1 -- 172.21.15.37:0/717349158 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb5ec063250 con 0x7fb5f80065e0 2024-09-17T04:33:26.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.152+0000 7fb2cf7fe640 1 --2- 172.21.15.37:0/4053142231 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb2bc078be0 0x7fb2bc07b0f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0035000 tx=0x7fb2c0038040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:26.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.152+0000 7efee6ffd640 1 --2- 172.21.15.37:0/2295979183 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7efed8078be0 0x7efed807b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.152+0000 7efee6ffd640 1 -- 172.21.15.37:0/2295979183 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7efee809a2b0 con 0x7efef0009ba0 2024-09-17T04:33:26.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.152+0000 7efef5d9b640 1 --2- 172.21.15.37:0/2295979183 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7efed8078be0 0x7efed807b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.153+0000 7efef5d9b640 1 --2- 172.21.15.37:0/2295979183 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7efed8078be0 0x7efed807b0f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efef00e3700 tx=0x7efef806cac0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:26.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.153+0000 7fb2b7fff640 1 -- 172.21.15.37:0/4053142231 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb2c8012560 con 0x7fb2d0151160 2024-09-17T04:33:26.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.153+0000 7ff7e6ffd640 1 -- 172.21.15.37:0/3323656362 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f00bb070 con 0x7ff7e8036400 2024-09-17T04:33:26.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.153+0000 7ff7e6ffd640 1 -- 172.21.15.37:0/3323656362 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7f00acc40 con 0x7ff7e8036400 2024-09-17T04:33:26.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.153+0000 7fac3e7fc640 1 -- 172.21.15.37:0/1717526960 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fac4004a060 con 0x7fac480cda50 2024-09-17T04:33:26.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.153+0000 7ff7e6ffd640 1 -- 172.21.15.37:0/3323656362 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f00c0460 con 0x7ff7e8036400 2024-09-17T04:33:26.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.154+0000 7ff7f7447640 1 -- 172.21.15.37:0/3323656362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7e8047cb0 con 0x7ff7e8036400 2024-09-17T04:33:26.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.154+0000 7ff7f7447640 1 -- 172.21.15.37:0/3323656362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7e801e790 con 0x7ff7e8036400 2024-09-17T04:33:26.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.155+0000 7ff7f7447640 1 -- 172.21.15.37:0/3323656362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7e8005cb0 con 0x7ff7e8036400 2024-09-17T04:33:26.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.159+0000 7fb2b7fff640 1 -- 172.21.15.37:0/4053142231 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb2c806b9d0 con 0x7fb2d0151160 2024-09-17T04:33:26.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.161+0000 7fe4a6ffd640 1 -- 172.21.15.37:0/2975744756 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe4a0083c50 con 0x7fe4a8073f70 2024-09-17T04:33:26.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.162+0000 7efee6ffd640 1 -- 172.21.15.37:0/2295979183 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7efee8062860 con 0x7efef0009ba0 2024-09-17T04:33:26.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.161+0000 7ff7e6ffd640 1 -- 172.21.15.37:0/3323656362 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff7f00b7410 con 0x7ff7e8036400 2024-09-17T04:33:26.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.162+0000 7ff7e6ffd640 1 --2- 172.21.15.37:0/3323656362 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff7d4078c60 0x7ff7d407b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:26.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.162+0000 7ff7e6ffd640 1 -- 172.21.15.37:0/3323656362 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff7f0137f80 con 0x7ff7e8036400 2024-09-17T04:33:26.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.163+0000 7ff7f6445640 1 --2- 172.21.15.37:0/3323656362 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff7d4078c60 0x7ff7d407b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:26.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.163+0000 7ff7f6445640 1 --2- 172.21.15.37:0/3323656362 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff7d4078c60 0x7ff7d407b170 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009890 tx=0x7ff7e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:26.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.169+0000 7ff7e6ffd640 1 -- 172.21.15.37:0/3323656362 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff7f01004c0 con 0x7ff7e8036400 2024-09-17T04:33:26.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.570+0000 7f9fbf7fe640 1 -- 172.21.15.37:0/2467013784 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f9fdc15d3e0 con 0x7f9fdc0af030 2024-09-17T04:33:26.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.571+0000 7f9fd1ffb640 1 -- 172.21.15.37:0/2467013784 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9fd40629e0 con 0x7f9fdc0af030 2024-09-17T04:33:26.571 INFO:teuthology.orchestra.run.smithi037.stdout:60129542149 2024-09-17T04:33:26.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.572+0000 7f132dcb2640 1 -- 172.21.15.37:0/1838546434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f1320008b20 con 0x7f13200dc0b0 2024-09-17T04:33:26.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.573+0000 7f1313fff640 1 -- 172.21.15.37:0/1838546434 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f13280df0d0 con 0x7f13200dc0b0 2024-09-17T04:33:26.573 INFO:teuthology.orchestra.run.smithi037.stdout:60129542150 2024-09-17T04:33:26.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.574+0000 7f9fbf7fe640 1 -- 172.21.15.37:0/2467013784 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9fc0078c90 msgr2=0x7f9fc007b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.574+0000 7f9fbf7fe640 1 --2- 172.21.15.37:0/2467013784 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9fc0078c90 0x7f9fc007b1a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9fd80095f0 tx=0x7f9fd8004730 comp rx=0 tx=0).stop 2024-09-17T04:33:26.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.574+0000 7f9fbf7fe640 1 -- 172.21.15.37:0/2467013784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fdc0af030 msgr2=0x7f9fdc2056e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.574+0000 7f9fbf7fe640 1 --2- 172.21.15.37:0/2467013784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fdc0af030 0x7f9fdc2056e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9fd400c730 tx=0x7f9fd400cbf0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.575+0000 7f9fbf7fe640 1 -- 172.21.15.37:0/2467013784 shutdown_connections 2024-09-17T04:33:26.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.575+0000 7f9fbf7fe640 1 --2- 172.21.15.37:0/2467013784 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9fc0078c90 0x7f9fc007b1a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.575+0000 7f9fbf7fe640 1 --2- 172.21.15.37:0/2467013784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fdc0af030 0x7f9fdc2056e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.575+0000 7f9fbf7fe640 1 --2- 172.21.15.37:0/2467013784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fdc0d0560 0x7f9fdc2051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.575+0000 7f9fbf7fe640 1 --2- 172.21.15.37:0/2467013784 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9fdc070070 0x7f9fdc0aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.575+0000 7f9fbf7fe640 1 -- 172.21.15.37:0/2467013784 >> 172.21.15.37:0/2467013784 conn(0x7f9fdc0b2bb0 msgr2=0x7f9fdc06cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:26.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.575+0000 7f9fbf7fe640 1 -- 172.21.15.37:0/2467013784 shutdown_connections 2024-09-17T04:33:26.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.575+0000 7f9fbf7fe640 1 -- 172.21.15.37:0/2467013784 wait complete. 2024-09-17T04:33:26.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.577+0000 7f1311ffb640 1 -- 172.21.15.37:0/1838546434 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f130c078c10 msgr2=0x7f130c07b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.577+0000 7f1311ffb640 1 --2- 172.21.15.37:0/1838546434 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f130c078c10 0x7f130c07b120 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f131c034000 tx=0x7f131c004730 comp rx=0 tx=0).stop 2024-09-17T04:33:26.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.578+0000 7f1311ffb640 1 -- 172.21.15.37:0/1838546434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13200dc0b0 msgr2=0x7f13200dc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.578+0000 7f1311ffb640 1 --2- 172.21.15.37:0/1838546434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13200dc0b0 0x7f13200dc4c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1328066830 tx=0x7f132806a0e0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.578+0000 7f1311ffb640 1 -- 172.21.15.37:0/1838546434 shutdown_connections 2024-09-17T04:33:26.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.578+0000 7f1311ffb640 1 --2- 172.21.15.37:0/1838546434 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f130c078c10 0x7f130c07b120 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.578+0000 7f1311ffb640 1 --2- 172.21.15.37:0/1838546434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13200dc0b0 0x7f13200dc4c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.578+0000 7f1311ffb640 1 --2- 172.21.15.37:0/1838546434 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f13200e0c10 0x7f13200dbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.578+0000 7f1311ffb640 1 --2- 172.21.15.37:0/1838546434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1320057560 0x7f13200d9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.578+0000 7f1311ffb640 1 -- 172.21.15.37:0/1838546434 >> 172.21.15.37:0/1838546434 conn(0x7f1320055e70 msgr2=0x7f132003dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:26.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.579+0000 7f1311ffb640 1 -- 172.21.15.37:0/1838546434 shutdown_connections 2024-09-17T04:33:26.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.579+0000 7f1311ffb640 1 -- 172.21.15.37:0/1838546434 wait complete. 2024-09-17T04:33:26.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.587+0000 7fb604c32640 1 -- 172.21.15.37:0/717349158 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fb5e0005580 con 0x7fb5f80065e0 2024-09-17T04:33:26.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.589+0000 7fb5ebfff640 1 -- 172.21.15.37:0/717349158 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb5ec067c80 con 0x7fb5f80065e0 2024-09-17T04:33:26.591 INFO:teuthology.orchestra.run.smithi037.stdout:60129542151 2024-09-17T04:33:26.592 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.1 2024-09-17T04:33:26.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.592+0000 7fb604c32640 1 -- 172.21.15.37:0/717349158 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb5e4078c90 msgr2=0x7fb5e407b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.592+0000 7fb604c32640 1 --2- 172.21.15.37:0/717349158 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb5e4078c90 0x7fb5e407b1a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4009590 tx=0x7fb5f4004730 comp rx=0 tx=0).stop 2024-09-17T04:33:26.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.592+0000 7fb604c32640 1 -- 172.21.15.37:0/717349158 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb5f80065e0 msgr2=0x7fb5f80dd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.592+0000 7fb604c32640 1 --2- 172.21.15.37:0/717349158 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb5f80065e0 0x7fb5f80dd420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00e6e0 tx=0x7fb5ec00eba0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.592+0000 7fb604c32640 1 -- 172.21.15.37:0/717349158 shutdown_connections 2024-09-17T04:33:26.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.592+0000 7fb604c32640 1 --2- 172.21.15.37:0/717349158 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb5e4078c90 0x7fb5e407b1a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.592+0000 7fb604c32640 1 --2- 172.21.15.37:0/717349158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5f80df390 0x7fb5f80dd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.592+0000 7fb604c32640 1 --2- 172.21.15.37:0/717349158 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb5f80065e0 0x7fb5f80dd420 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.592+0000 7fb604c32640 1 --2- 172.21.15.37:0/717349158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5f8004a70 0x7fb5f80ed8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.592+0000 7fb604c32640 1 -- 172.21.15.37:0/717349158 >> 172.21.15.37:0/717349158 conn(0x7fb5f801ddf0 msgr2=0x7fb5f80069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:26.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.592+0000 7fb604c32640 1 -- 172.21.15.37:0/717349158 shutdown_connections 2024-09-17T04:33:26.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.592+0000 7fb604c32640 1 -- 172.21.15.37:0/717349158 wait complete. 2024-09-17T04:33:26.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.595+0000 7fac50157640 1 -- 172.21.15.37:0/1717526960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fac480ae1c0 con 0x7fac480cda50 2024-09-17T04:33:26.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.595+0000 7fac3e7fc640 1 -- 172.21.15.37:0/1717526960 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fac40084210 con 0x7fac480cda50 2024-09-17T04:33:26.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.595+0000 7efee4ff9640 1 -- 172.21.15.37:0/2295979183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7efed0005850 con 0x7efef0009ba0 2024-09-17T04:33:26.595 INFO:teuthology.orchestra.run.smithi037.stdout:60129542150 2024-09-17T04:33:26.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.596+0000 7efee6ffd640 1 -- 172.21.15.37:0/2295979183 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efee8062160 con 0x7efef0009ba0 2024-09-17T04:33:26.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.596+0000 7fe4b0494640 1 -- 172.21.15.37:0/2975744756 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fe488005580 con 0x7fe4a8073f70 2024-09-17T04:33:26.596 INFO:teuthology.orchestra.run.smithi037.stdout:60129542150 2024-09-17T04:33:26.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.597+0000 7fe4a6ffd640 1 -- 172.21.15.37:0/2975744756 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe4a0083550 con 0x7fe4a8073f70 2024-09-17T04:33:26.597 INFO:teuthology.orchestra.run.smithi037.stdout:60129542150 2024-09-17T04:33:26.598 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.6 2024-09-17T04:33:26.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.597+0000 7fb2d6151640 1 -- 172.21.15.37:0/4053142231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fb2d00b08a0 con 0x7fb2d0151160 2024-09-17T04:33:26.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.598+0000 7fb2b7fff640 1 -- 172.21.15.37:0/4053142231 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb2c806b2d0 con 0x7fb2d0151160 2024-09-17T04:33:26.598 INFO:teuthology.orchestra.run.smithi037.stdout:60129542151 2024-09-17T04:33:26.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.599+0000 7fac50157640 1 -- 172.21.15.37:0/1717526960 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fac2c078c60 msgr2=0x7fac2c07b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.599+0000 7fac50157640 1 --2- 172.21.15.37:0/1717526960 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fac2c078c60 0x7fac2c07b170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fac38009b30 tx=0x7fac38005f90 comp rx=0 tx=0).stop 2024-09-17T04:33:26.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.599+0000 7fac50157640 1 -- 172.21.15.37:0/1717526960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac480cda50 msgr2=0x7fac481fd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.599+0000 7fac50157640 1 --2- 172.21.15.37:0/1717526960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac480cda50 0x7fac481fd3c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fac40032000 tx=0x7fac40003f10 comp rx=0 tx=0).stop 2024-09-17T04:33:26.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.599+0000 7efee4ff9640 1 -- 172.21.15.37:0/2295979183 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7efed8078be0 msgr2=0x7efed807b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.599+0000 7efee4ff9640 1 --2- 172.21.15.37:0/2295979183 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7efed8078be0 0x7efed807b0f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efef00e3700 tx=0x7efef806cac0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.599+0000 7efee4ff9640 1 -- 172.21.15.37:0/2295979183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efef0009ba0 msgr2=0x7efef00de8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.599+0000 7efee4ff9640 1 --2- 172.21.15.37:0/2295979183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efef0009ba0 0x7efef00de8e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efee800c420 tx=0x7efee800c8e0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.599+0000 7fac50157640 1 -- 172.21.15.37:0/1717526960 shutdown_connections 2024-09-17T04:33:26.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.599+0000 7efee4ff9640 1 -- 172.21.15.37:0/2295979183 shutdown_connections 2024-09-17T04:33:26.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.599+0000 7efee4ff9640 1 --2- 172.21.15.37:0/2295979183 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7efed8078be0 0x7efed807b0f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.599+0000 7efee4ff9640 1 --2- 172.21.15.37:0/2295979183 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7efef0036b10 0x7efef017e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.599+0000 7efee4ff9640 1 --2- 172.21.15.37:0/2295979183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efef000bc30 0x7efef017db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.599+0000 7efee4ff9640 1 --2- 172.21.15.37:0/2295979183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efef0009ba0 0x7efef00de8e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.599+0000 7efee4ff9640 1 -- 172.21.15.37:0/2295979183 >> 172.21.15.37:0/2295979183 conn(0x7efef0051420 msgr2=0x7efef0038410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:26.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.599+0000 7fac50157640 1 --2- 172.21.15.37:0/1717526960 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fac2c078c60 0x7fac2c07b170 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.599+0000 7fac50157640 1 --2- 172.21.15.37:0/1717526960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac480cda50 0x7fac481fd3c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.599+0000 7fac50157640 1 --2- 172.21.15.37:0/1717526960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac480b6640 0x7fac4814df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.599+0000 7fac50157640 1 --2- 172.21.15.37:0/1717526960 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fac480b0de0 0x7fac48155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.599+0000 7fac50157640 1 -- 172.21.15.37:0/1717526960 >> 172.21.15.37:0/1717526960 conn(0x7fac4809aa20 msgr2=0x7fac48070a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:26.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.599+0000 7efee4ff9640 1 -- 172.21.15.37:0/2295979183 shutdown_connections 2024-09-17T04:33:26.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.599+0000 7efee4ff9640 1 -- 172.21.15.37:0/2295979183 wait complete. 2024-09-17T04:33:26.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.599+0000 7fac50157640 1 -- 172.21.15.37:0/1717526960 shutdown_connections 2024-09-17T04:33:26.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.599+0000 7fac50157640 1 -- 172.21.15.37:0/1717526960 wait complete. 2024-09-17T04:33:26.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.600+0000 7fe4b0494640 1 -- 172.21.15.37:0/2975744756 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe494078c10 msgr2=0x7fe49407b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.600+0000 7fe4b0494640 1 --2- 172.21.15.37:0/2975744756 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe494078c10 0x7fe49407b120 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe4a80cce80 tx=0x7fe498009500 comp rx=0 tx=0).stop 2024-09-17T04:33:26.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.600+0000 7fe4b0494640 1 -- 172.21.15.37:0/2975744756 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe4a8073f70 msgr2=0x7fe4a80b4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.600+0000 7fe4b0494640 1 --2- 172.21.15.37:0/2975744756 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe4a8073f70 0x7fe4a80b4290 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0034040 tx=0x7fe4a0003b90 comp rx=0 tx=0).stop 2024-09-17T04:33:26.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.600+0000 7fe4b0494640 1 -- 172.21.15.37:0/2975744756 shutdown_connections 2024-09-17T04:33:26.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.600+0000 7fe4b0494640 1 --2- 172.21.15.37:0/2975744756 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe494078c10 0x7fe49407b120 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.600+0000 7fe4b0494640 1 --2- 172.21.15.37:0/2975744756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a80c5da0 0x7fe4a80c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.600+0000 7fe4b0494640 1 --2- 172.21.15.37:0/2975744756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4a8075b90 0x7fe4a80c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.600+0000 7fe4b0494640 1 --2- 172.21.15.37:0/2975744756 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe4a8073f70 0x7fe4a80b4290 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.600+0000 7fe4b0494640 1 -- 172.21.15.37:0/2975744756 >> 172.21.15.37:0/2975744756 conn(0x7fe4a80b20e0 msgr2=0x7fe4a80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:26.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.601+0000 7fe4b0494640 1 -- 172.21.15.37:0/2975744756 shutdown_connections 2024-09-17T04:33:26.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.601+0000 7fe4b0494640 1 -- 172.21.15.37:0/2975744756 wait complete. 2024-09-17T04:33:26.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.601+0000 7fb2d6151640 1 -- 172.21.15.37:0/4053142231 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb2bc078be0 msgr2=0x7fb2bc07b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.601+0000 7fb2d6151640 1 --2- 172.21.15.37:0/4053142231 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb2bc078be0 0x7fb2bc07b0f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0035000 tx=0x7fb2c0038040 comp rx=0 tx=0).stop 2024-09-17T04:33:26.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.601+0000 7fb2d6151640 1 -- 172.21.15.37:0/4053142231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2d0151160 msgr2=0x7fb2d01f4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.601+0000 7fb2d6151640 1 --2- 172.21.15.37:0/4053142231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2d0151160 0x7fb2d01f4ef0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8014300 tx=0x7fb2c8014700 comp rx=0 tx=0).stop 2024-09-17T04:33:26.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.602+0000 7fb2d6151640 1 -- 172.21.15.37:0/4053142231 shutdown_connections 2024-09-17T04:33:26.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.602+0000 7fb2d6151640 1 --2- 172.21.15.37:0/4053142231 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb2bc078be0 0x7fb2bc07b0f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.602+0000 7fb2d6151640 1 --2- 172.21.15.37:0/4053142231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2d0151160 0x7fb2d01f4ef0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.602+0000 7fb2d6151640 1 --2- 172.21.15.37:0/4053142231 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb2d00ccae0 0x7fb2d01f49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.602+0000 7fb2d6151640 1 --2- 172.21.15.37:0/4053142231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2d006f480 0x7fb2d0074360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.602+0000 7fb2d6151640 1 -- 172.21.15.37:0/4053142231 >> 172.21.15.37:0/4053142231 conn(0x7fb2d00b1ca0 msgr2=0x7fb2d015c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:26.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.602+0000 7fb2d6151640 1 -- 172.21.15.37:0/4053142231 shutdown_connections 2024-09-17T04:33:26.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.602+0000 7fb2d6151640 1 -- 172.21.15.37:0/4053142231 wait complete. 2024-09-17T04:33:26.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.603+0000 7ff7f7447640 1 -- 172.21.15.37:0/3323656362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff7e8047a40 con 0x7ff7e8036400 2024-09-17T04:33:26.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.603+0000 7ff7e6ffd640 1 -- 172.21.15.37:0/3323656362 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff7f00ffdc0 con 0x7ff7e8036400 2024-09-17T04:33:26.606 INFO:teuthology.orchestra.run.smithi037.stdout:60129542150 2024-09-17T04:33:26.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.606+0000 7ff7f7447640 1 -- 172.21.15.37:0/3323656362 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff7d4078c60 msgr2=0x7ff7d407b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.606+0000 7ff7f7447640 1 --2- 172.21.15.37:0/3323656362 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff7d4078c60 0x7ff7d407b170 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009890 tx=0x7ff7e0008040 comp rx=0 tx=0).stop 2024-09-17T04:33:26.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.606+0000 7ff7f7447640 1 -- 172.21.15.37:0/3323656362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7e8036400 msgr2=0x7ff7e80209a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:26.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.606+0000 7ff7f7447640 1 --2- 172.21.15.37:0/3323656362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7e8036400 0x7ff7e80209a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0052860 tx=0x7ff7f00ac5d0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.606+0000 7ff7f7447640 1 -- 172.21.15.37:0/3323656362 shutdown_connections 2024-09-17T04:33:26.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.606+0000 7ff7f7447640 1 --2- 172.21.15.37:0/3323656362 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff7d4078c60 0x7ff7d407b170 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.606+0000 7ff7f7447640 1 --2- 172.21.15.37:0/3323656362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7e8036400 0x7ff7e80209a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.606+0000 7ff7f7447640 1 --2- 172.21.15.37:0/3323656362 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff7e800bc30 0x7ff7e8020460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.606+0000 7ff7f7447640 1 --2- 172.21.15.37:0/3323656362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7e8009ba0 0x7ff7e80defa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:26.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.606+0000 7ff7f7447640 1 -- 172.21.15.37:0/3323656362 >> 172.21.15.37:0/3323656362 conn(0x7ff7e8051420 msgr2=0x7ff7e8034e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:26.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.606+0000 7ff7f7447640 1 -- 172.21.15.37:0/3323656362 shutdown_connections 2024-09-17T04:33:26.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:26.606+0000 7ff7f7447640 1 -- 172.21.15.37:0/3323656362 wait complete. 2024-09-17T04:33:26.616 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.4 2024-09-17T04:33:26.617 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.5 2024-09-17T04:33:26.620 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.2 2024-09-17T04:33:26.621 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.0 2024-09-17T04:33:26.622 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.7 2024-09-17T04:33:26.623 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.3 2024-09-17T04:33:27.592 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T04:33:27.598 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T04:33:27.618 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T04:33:27.618 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T04:33:27.621 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T04:33:27.622 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T04:33:27.622 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T04:33:27.624 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T04:33:27.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.797+0000 7f6f75e8b640 1 -- 172.21.15.37:0/3662643393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f7015a900 msgr2=0x7f6f7006f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.797+0000 7f6f75e8b640 1 --2- 172.21.15.37:0/3662643393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f7015a900 0x7f6f7006f4f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6f700d1180 tx=0x7f6f6402a4b0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.798+0000 7f6f75e8b640 1 -- 172.21.15.37:0/3662643393 shutdown_connections 2024-09-17T04:33:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.798+0000 7f6f75e8b640 1 --2- 172.21.15.37:0/3662643393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f7015a900 0x7f6f7006f4f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.798+0000 7f6f75e8b640 1 --2- 172.21.15.37:0/3662643393 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6f70159fb0 0x7f6f7015a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.798+0000 7f6f75e8b640 1 -- 172.21.15.37:0/3662643393 >> 172.21.15.37:0/3662643393 conn(0x7f6f700d00d0 msgr2=0x7f6f700d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.799+0000 7f6f75e8b640 1 -- 172.21.15.37:0/3662643393 shutdown_connections 2024-09-17T04:33:27.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.799+0000 7f6f75e8b640 1 -- 172.21.15.37:0/3662643393 wait complete. 2024-09-17T04:33:27.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.802+0000 7f6f75e8b640 1 Processor -- start 2024-09-17T04:33:27.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.803+0000 7f6f75e8b640 1 -- start start 2024-09-17T04:33:27.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.805+0000 7f6f75e8b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f700d0bf0 0x7f6f700c4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.806+0000 7f6f75e8b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6f70159fb0 0x7f6f701567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.806+0000 7f6f75e8b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f7015a900 0x7f6f7009bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.806+0000 7f6f6f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f700d0bf0 0x7f6f700c4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.806+0000 7f6f6f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f700d0bf0 0x7f6f700c4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.37:50806/0 (socket says 172.21.15.37:50806) 2024-09-17T04:33:27.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.806+0000 7f6f6effd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6f70159fb0 0x7f6f701567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.806+0000 7f6f6effd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6f70159fb0 0x7f6f701567f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.37:38814/0 (socket says 172.21.15.37:38814) 2024-09-17T04:33:27.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.807+0000 7f6f6ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f7015a900 0x7f6f7009bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.807+0000 7f6f6ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f7015a900 0x7f6f7009bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45796/0 (socket says 172.21.15.37:45796) 2024-09-17T04:33:27.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.807+0000 7f6f6f7fe640 1 -- 172.21.15.37:0/4216776464 learned_addr learned my addr 172.21.15.37:0/4216776464 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:27.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.807+0000 7f6f75e8b640 1 -- 172.21.15.37:0/4216776464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6f70073b10 con 0x7f6f7015a900 2024-09-17T04:33:27.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.807+0000 7f6f75e8b640 1 -- 172.21.15.37:0/4216776464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6f70073990 con 0x7f6f700d0bf0 2024-09-17T04:33:27.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.807+0000 7f6f75e8b640 1 -- 172.21.15.37:0/4216776464 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f6f7006fa30 con 0x7f6f70159fb0 2024-09-17T04:33:27.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.808+0000 7f6f6effd640 1 -- 172.21.15.37:0/4216776464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f700d0bf0 msgr2=0x7f6f700c4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:27.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.808+0000 7f6f6effd640 1 --2- 172.21.15.37:0/4216776464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f700d0bf0 0x7f6f700c4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.808+0000 7f6f6effd640 1 -- 172.21.15.37:0/4216776464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f7015a900 msgr2=0x7f6f7009bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:27.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.808+0000 7f6f6effd640 1 --2- 172.21.15.37:0/4216776464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f7015a900 0x7f6f7009bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.808+0000 7f6f6effd640 1 -- 172.21.15.37:0/4216776464 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f7009b710 con 0x7f6f70159fb0 2024-09-17T04:33:27.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.809+0000 7f6f6effd640 1 --2- 172.21.15.37:0/4216776464 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6f70159fb0 0x7f6f701567f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000e720 tx=0x7f6f6000ebe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:27.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.809+0000 7f6f6ffff640 1 --2- 172.21.15.37:0/4216776464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f7015a900 0x7f6f7009bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:33:27.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.809+0000 7f6f57fff640 1 -- 172.21.15.37:0/4216776464 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60004220 con 0x7f6f70159fb0 2024-09-17T04:33:27.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.810+0000 7f6f75e8b640 1 -- 172.21.15.37:0/4216776464 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f700c3bf0 con 0x7f6f70159fb0 2024-09-17T04:33:27.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.810+0000 7f6f75e8b640 1 -- 172.21.15.37:0/4216776464 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f7009c690 con 0x7f6f70159fb0 2024-09-17T04:33:27.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.810+0000 7f6f57fff640 1 -- 172.21.15.37:0/4216776464 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f6000c8b0 con 0x7f6f70159fb0 2024-09-17T04:33:27.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.810+0000 7f6f57fff640 1 -- 172.21.15.37:0/4216776464 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6000cd00 con 0x7f6f70159fb0 2024-09-17T04:33:27.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.815+0000 7f6f57fff640 1 -- 172.21.15.37:0/4216776464 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6f600035f0 con 0x7f6f70159fb0 2024-09-17T04:33:27.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.815+0000 7f6f57fff640 1 --2- 172.21.15.37:0/4216776464 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6f58078d80 0x7f6f5807b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.815+0000 7f6f57fff640 1 -- 172.21.15.37:0/4216776464 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6f6009b000 con 0x7f6f70159fb0 2024-09-17T04:33:27.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.816+0000 7f6f75e8b640 1 -- 172.21.15.37:0/4216776464 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f50005290 con 0x7f6f70159fb0 2024-09-17T04:33:27.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.817+0000 7f6f6f7fe640 1 --2- 172.21.15.37:0/4216776464 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6f58078d80 0x7f6f5807b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.817+0000 7f6f6f7fe640 1 --2- 172.21.15.37:0/4216776464 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6f58078d80 0x7f6f5807b290 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6f680012e0 tx=0x7f6f68009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:27.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.822+0000 7f6f57fff640 1 -- 172.21.15.37:0/4216776464 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6f60062e60 con 0x7f6f70159fb0 2024-09-17T04:33:27.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.859+0000 7f318d358640 1 -- 172.21.15.37:0/3463648725 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3188070a80 msgr2=0x7f318815f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:27.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.859+0000 7f318d358640 1 --2- 172.21.15.37:0/3463648725 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3188070a80 0x7f318815f560 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3178009be0 tx=0x7f317802a740 comp rx=0 tx=0).stop 2024-09-17T04:33:27.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.859+0000 7f318d358640 1 -- 172.21.15.37:0/3463648725 shutdown_connections 2024-09-17T04:33:27.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.859+0000 7f318d358640 1 --2- 172.21.15.37:0/3463648725 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3188070a80 0x7f318815f560 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.859+0000 7f318d358640 1 -- 172.21.15.37:0/3463648725 >> 172.21.15.37:0/3463648725 conn(0x7f31880b2000 msgr2=0x7f31880b2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:27.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.860+0000 7f318d358640 1 -- 172.21.15.37:0/3463648725 shutdown_connections 2024-09-17T04:33:27.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.860+0000 7f6d7160e640 1 -- 172.21.15.37:0/2908659267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d64009ba0 msgr2=0x7f6d640df180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:27.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.860+0000 7f6d7160e640 1 --2- 172.21.15.37:0/2908659267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d64009ba0 0x7f6d640df180 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c07ab50 tx=0x7f6d6c0a83b0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.860+0000 7f6d7160e640 1 -- 172.21.15.37:0/2908659267 shutdown_connections 2024-09-17T04:33:27.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.860+0000 7f6d7160e640 1 --2- 172.21.15.37:0/2908659267 >> v2:172.21.15.92:3300/0 conn(0x7f6d64036b40 0x7f6d640066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.860+0000 7f6d7160e640 1 --2- 172.21.15.37:0/2908659267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d64009ba0 0x7f6d640df180 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.860+0000 7f6d7160e640 1 -- 172.21.15.37:0/2908659267 >> 172.21.15.37:0/2908659267 conn(0x7f6d64051420 msgr2=0x7f6d64051860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:27.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.860+0000 7fbcddebe640 1 -- 172.21.15.37:0/3028547285 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbcd0007680 msgr2=0x7fbcd00df1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:27.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.860+0000 7fbcddebe640 1 --2- 172.21.15.37:0/3028547285 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbcd0007680 0x7fbcd00df1b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80774d0 tx=0x7fbcd80a6430 comp rx=0 tx=0).stop 2024-09-17T04:33:27.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.860+0000 7fbcddebe640 1 -- 172.21.15.37:0/3028547285 shutdown_connections 2024-09-17T04:33:27.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.860+0000 7fbcddebe640 1 --2- 172.21.15.37:0/3028547285 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbcd0007680 0x7fbcd00df1b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.860+0000 7fbcddebe640 1 -- 172.21.15.37:0/3028547285 >> 172.21.15.37:0/3028547285 conn(0x7fbcd001ddf0 msgr2=0x7fbcd0039ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:27.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.860+0000 7fbcddebe640 1 -- 172.21.15.37:0/3028547285 shutdown_connections 2024-09-17T04:33:27.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.861+0000 7fbcddebe640 1 -- 172.21.15.37:0/3028547285 wait complete. 2024-09-17T04:33:27.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.860+0000 7f6d7160e640 1 -- 172.21.15.37:0/2908659267 shutdown_connections 2024-09-17T04:33:27.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.860+0000 7f6d7160e640 1 -- 172.21.15.37:0/2908659267 wait complete. 2024-09-17T04:33:27.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.863+0000 7f6d7160e640 1 Processor -- start 2024-09-17T04:33:27.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.864+0000 7f6d7160e640 1 -- start start 2024-09-17T04:33:27.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.864+0000 7fbcddebe640 1 Processor -- start 2024-09-17T04:33:27.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.860+0000 7f318d358640 1 -- 172.21.15.37:0/3463648725 wait complete. 2024-09-17T04:33:27.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.863+0000 7f318d358640 1 Processor -- start 2024-09-17T04:33:27.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.863+0000 7f318d358640 1 -- start start 2024-09-17T04:33:27.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.864+0000 7fbcddebe640 1 -- start start 2024-09-17T04:33:27.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.866+0000 7f318d358640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3188070a80 0x7f3188209f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.866+0000 7f318d358640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3188073710 0x7f31880adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.866+0000 7f6d7160e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d64005cb0 0x7f6d640de580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.866+0000 7f3187fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3188070a80 0x7f3188209f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.866+0000 7f3187fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3188070a80 0x7f3188209f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.37:50838/0 (socket says 172.21.15.37:50838) 2024-09-17T04:33:27.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.866+0000 7f3187fff640 1 -- 172.21.15.37:0/1196631736 learned_addr learned my addr 172.21.15.37:0/1196631736 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:27.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.866+0000 7f318d358640 1 --2- 172.21.15.37:0/1196631736 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f31880b2f10 0x7f31880cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.866+0000 7f31877fe640 1 --2- 172.21.15.37:0/1196631736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3188073710 0x7f31880adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.866+0000 7f6d6affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d64005cb0 0x7f6d640de580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.866+0000 7f318d358640 1 -- 172.21.15.37:0/1196631736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f318806f6e0 con 0x7f3188073710 2024-09-17T04:33:27.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.867+0000 7fbcddebe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcd0004a70 0x7fbcd00d0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.866+0000 7f318d358640 1 -- 172.21.15.37:0/1196631736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f318806f560 con 0x7f3188070a80 2024-09-17T04:33:27.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.867+0000 7f6d6affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d64005cb0 0x7f6d640de580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45830/0 (socket says 172.21.15.37:45830) 2024-09-17T04:33:27.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.867+0000 7f6d7160e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6d6418a100 0x7f6d6418a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.867+0000 7f318d358640 1 -- 172.21.15.37:0/1196631736 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f3188155160 con 0x7f31880b2f10 2024-09-17T04:33:27.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.867+0000 7fbcddebe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcd0007680 0x7fbcd0032cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.867+0000 7f318cb57640 1 --2- 172.21.15.37:0/1196631736 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f31880b2f10 0x7f31880cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.867+0000 7fbcddebe640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbcd0032360 0x7fbcd0033200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.867+0000 7fbcdcebc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcd0004a70 0x7fbcd00d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.867+0000 7fbcddebe640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd003a340 con 0x7fbcd0004a70 2024-09-17T04:33:27.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.867+0000 7fbcddebe640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd003a1c0 con 0x7fbcd0007680 2024-09-17T04:33:27.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.867+0000 7fbcddebe640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fbcd00e3490 con 0x7fbcd0032360 2024-09-17T04:33:27.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.867+0000 7fbcdcebc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcd0004a70 0x7fbcd00d0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45846/0 (socket says 172.21.15.37:45846) 2024-09-17T04:33:27.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.867+0000 7fbcdd6bd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbcd0032360 0x7fbcd0033200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.867+0000 7fbcdcebc640 1 -- 172.21.15.37:0/2143305204 learned_addr learned my addr 172.21.15.37:0/2143305204 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:27.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.867+0000 7fbcd7fff640 1 --2- 172.21.15.37:0/2143305204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcd0007680 0x7fbcd0032cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.867+0000 7f318cb57640 1 -- 172.21.15.37:0/1196631736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3188070a80 msgr2=0x7f3188209f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:27.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.867+0000 7f318cb57640 1 --2- 172.21.15.37:0/1196631736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3188070a80 0x7f3188209f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.867+0000 7f318cb57640 1 -- 172.21.15.37:0/1196631736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3188073710 msgr2=0x7f31880adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:27.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.867+0000 7f318cb57640 1 --2- 172.21.15.37:0/1196631736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3188073710 0x7f31880adf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.867+0000 7f318cb57640 1 -- 172.21.15.37:0/1196631736 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318820aac0 con 0x7f31880b2f10 2024-09-17T04:33:27.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.867+0000 7f31877fe640 1 --2- 172.21.15.37:0/1196631736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3188073710 0x7f31880adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:27.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.868+0000 7f318cb57640 1 --2- 172.21.15.37:0/1196631736 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f31880b2f10 0x7f31880cfd50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f318000c390 tx=0x7f318000c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:27.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.868+0000 7f0995ffb640 1 -- 172.21.15.37:0/2659990043 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099002bcb0 con 0x7f09980cc8b0 2024-09-17T04:33:27.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.867+0000 7f6d7160e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d64021a50 0x7f6d64021e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.867+0000 7f6d7160e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6400be40 con 0x7f6d64005cb0 2024-09-17T04:33:27.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.867+0000 7f6d7160e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6d64006bf0 con 0x7f6d64021a50 2024-09-17T04:33:27.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.867+0000 7f6d7160e640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f6d64172960 con 0x7f6d6418a100 2024-09-17T04:33:27.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.867+0000 7f6d6b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d64021a50 0x7f6d64021e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.868+0000 7f6d6affd640 1 -- 172.21.15.37:0/120885470 learned_addr learned my addr 172.21.15.37:0/120885470 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:27.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.868+0000 7f099d72b640 1 -- 172.21.15.37:0/2659990043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09980cc8b0 msgr2=0x7f0998161ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:27.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.868+0000 7f099d72b640 1 --2- 172.21.15.37:0/2659990043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09980cc8b0 0x7f0998161ea0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0990005fd0 tx=0x7f099002ac10 comp rx=0 tx=0).stop 2024-09-17T04:33:27.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.868+0000 7f099d72b640 1 -- 172.21.15.37:0/2659990043 shutdown_connections 2024-09-17T04:33:27.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.868+0000 7f099d72b640 1 --2- 172.21.15.37:0/2659990043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09980cc8b0 0x7f0998161ea0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.868+0000 7f099d72b640 1 -- 172.21.15.37:0/2659990043 >> 172.21.15.37:0/2659990043 conn(0x7f099809aa20 msgr2=0x7f09980b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:27.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.868+0000 7fbcdd6bd640 1 -- 172.21.15.37:0/2143305204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcd0007680 msgr2=0x7fbcd0032cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:27.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.868+0000 7fbcdd6bd640 1 --2- 172.21.15.37:0/2143305204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcd0007680 0x7fbcd0032cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.868+0000 7fbcdd6bd640 1 -- 172.21.15.37:0/2143305204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcd0004a70 msgr2=0x7fbcd00d0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:27.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.868+0000 7fbcdd6bd640 1 --2- 172.21.15.37:0/2143305204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcd0004a70 0x7fbcd00d0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.868+0000 7fbcdd6bd640 1 -- 172.21.15.37:0/2143305204 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd00db0e0 con 0x7fbcd0032360 2024-09-17T04:33:27.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.868+0000 7fbcdd6bd640 1 --2- 172.21.15.37:0/2143305204 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbcd0032360 0x7fbcd0033200 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8051d10 tx=0x7fbcd80adcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:27.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.869+0000 7f64aff20640 1 -- 172.21.15.37:0/2579135529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64a00e48f0 msgr2=0x7f64a00533e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:27.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.869+0000 7f64aff20640 1 --2- 172.21.15.37:0/2579135529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64a00e48f0 0x7f64a00533e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f649c0098e0 tx=0x7f649c02a4b0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.868+0000 7f3184ff9640 1 -- 172.21.15.37:0/1196631736 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3180017070 con 0x7f31880b2f10 2024-09-17T04:33:27.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.868+0000 7f3184ff9640 1 -- 172.21.15.37:0/1196631736 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31800035f0 con 0x7f31880b2f10 2024-09-17T04:33:27.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.868+0000 7f3184ff9640 1 -- 172.21.15.37:0/1196631736 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3180003a40 con 0x7f31880b2f10 2024-09-17T04:33:27.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.869+0000 7f318d358640 1 -- 172.21.15.37:0/1196631736 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f318820ada0 con 0x7f31880b2f10 2024-09-17T04:33:27.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.869+0000 7f318d358640 1 -- 172.21.15.37:0/1196631736 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f318820b360 con 0x7f31880b2f10 2024-09-17T04:33:27.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.869+0000 7fbcd57fa640 1 -- 172.21.15.37:0/2143305204 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd80bb070 con 0x7fbcd0032360 2024-09-17T04:33:27.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.869+0000 7fbcd57fa640 1 -- 172.21.15.37:0/2143305204 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcd80b7910 con 0x7fbcd0032360 2024-09-17T04:33:27.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.869+0000 7fbcd57fa640 1 -- 172.21.15.37:0/2143305204 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd80b7cc0 con 0x7fbcd0032360 2024-09-17T04:33:27.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.869+0000 7f6d6a7fc640 1 --2- 172.21.15.37:0/120885470 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6d6418a100 0x7f6d6418a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.869+0000 7f6d6affd640 1 -- 172.21.15.37:0/120885470 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6d6418a100 msgr2=0x7f6d6418a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:27.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.869+0000 7f6d6affd640 1 --2- 172.21.15.37:0/120885470 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6d6418a100 0x7f6d6418a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.869+0000 7f6d6affd640 1 -- 172.21.15.37:0/120885470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d64021a50 msgr2=0x7f6d64021e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.869+0000 7f6d6affd640 1 --2- 172.21.15.37:0/120885470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d64021a50 0x7f6d64021e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.869+0000 7f6d6affd640 1 -- 172.21.15.37:0/120885470 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6418aa50 con 0x7f6d64005cb0 2024-09-17T04:33:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.869+0000 7fbcddebe640 1 -- 172.21.15.37:0/2143305204 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd00db380 con 0x7fbcd0032360 2024-09-17T04:33:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.869+0000 7fbcddebe640 1 -- 172.21.15.37:0/2143305204 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd00ec800 con 0x7fbcd0032360 2024-09-17T04:33:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.869+0000 7fa97f0f6640 1 -- 172.21.15.37:0/3392356400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa978071cd0 msgr2=0x7fa9780cfa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:27.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.869+0000 7fa97f0f6640 1 --2- 172.21.15.37:0/3392356400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa978071cd0 0x7fa9780cfa60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa970009ca0 tx=0x7fa97002aac0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.869+0000 7f6d6affd640 1 --2- 172.21.15.37:0/120885470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d64005cb0 0x7f6d640de580 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c077560 tx=0x7f6d6c0b1680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:27.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.870+0000 7f099d72b640 1 -- 172.21.15.37:0/2659990043 shutdown_connections 2024-09-17T04:33:27.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.870+0000 7f6d537fe640 1 -- 172.21.15.37:0/120885470 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c0b6070 con 0x7f6d64005cb0 2024-09-17T04:33:27.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.870+0000 7f6d537fe640 1 -- 172.21.15.37:0/120885470 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d6c0b1c80 con 0x7f6d64005cb0 2024-09-17T04:33:27.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.870+0000 7f6d537fe640 1 -- 172.21.15.37:0/120885470 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c0bb460 con 0x7f6d64005cb0 2024-09-17T04:33:27.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.870+0000 7f099d72b640 1 -- 172.21.15.37:0/2659990043 wait complete. 2024-09-17T04:33:27.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.870+0000 7f318d358640 1 -- 172.21.15.37:0/1196631736 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f318820b560 con 0x7f31880b2f10 2024-09-17T04:33:27.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.870+0000 7f64aff20640 1 -- 172.21.15.37:0/2579135529 shutdown_connections 2024-09-17T04:33:27.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.870+0000 7f64aff20640 1 --2- 172.21.15.37:0/2579135529 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f64a00e0b30 0x7f64a0037520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.870+0000 7f64aff20640 1 --2- 172.21.15.37:0/2579135529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64a00e48f0 0x7f64a00533e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.870+0000 7f64aff20640 1 -- 172.21.15.37:0/2579135529 >> 172.21.15.37:0/2579135529 conn(0x7f64a001eaf0 msgr2=0x7f64a003a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:27.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.871+0000 7f64aff20640 1 -- 172.21.15.37:0/2579135529 shutdown_connections 2024-09-17T04:33:27.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.871+0000 7f64aff20640 1 -- 172.21.15.37:0/2579135529 wait complete. 2024-09-17T04:33:27.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.871+0000 7f1752f07640 1 -- 172.21.15.37:0/1806282560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17440356e0 msgr2=0x7f1744053440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:27.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.870+0000 7fa97f0f6640 1 -- 172.21.15.37:0/3392356400 shutdown_connections 2024-09-17T04:33:27.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.870+0000 7fa97f0f6640 1 --2- 172.21.15.37:0/3392356400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa978071cd0 0x7fa9780cfa60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.870+0000 7fa97f0f6640 1 --2- 172.21.15.37:0/3392356400 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa97815f520 0x7fa978070b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.870+0000 7fa97f0f6640 1 -- 172.21.15.37:0/3392356400 >> 172.21.15.37:0/3392356400 conn(0x7fa9780b2bb0 msgr2=0x7fa9780b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:27.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.870+0000 7fa97f0f6640 1 -- 172.21.15.37:0/3392356400 shutdown_connections 2024-09-17T04:33:27.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.870+0000 7fa97f0f6640 1 -- 172.21.15.37:0/3392356400 wait complete. 2024-09-17T04:33:27.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.871+0000 7f1752f07640 1 --2- 172.21.15.37:0/1806282560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17440356e0 0x7f1744053440 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f174c06ae30 tx=0x7f174c0a6390 comp rx=0 tx=0).stop 2024-09-17T04:33:27.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.871+0000 7f1752f07640 1 -- 172.21.15.37:0/1806282560 shutdown_connections 2024-09-17T04:33:27.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.871+0000 7f6d7160e640 1 -- 172.21.15.37:0/120885470 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d640223a0 con 0x7f6d64005cb0 2024-09-17T04:33:27.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.871+0000 7f1752f07640 1 --2- 172.21.15.37:0/1806282560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17440356e0 0x7f1744053440 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.871+0000 7f1752f07640 1 --2- 172.21.15.37:0/1806282560 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1744055130 0x7f17440351a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.871+0000 7f1752f07640 1 -- 172.21.15.37:0/1806282560 >> 172.21.15.37:0/1806282560 conn(0x7f174401eac0 msgr2=0x7f174403a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:27.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.872+0000 7fbcc2ffd640 1 -- 172.21.15.37:0/2143305204 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcd00e39d0 con 0x7fbcd0032360 2024-09-17T04:33:27.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.872+0000 7f6d7160e640 1 -- 172.21.15.37:0/120885470 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d64172e90 con 0x7f6d64005cb0 2024-09-17T04:33:27.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.873+0000 7f6d517fa640 1 -- 172.21.15.37:0/120885470 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d6402aec0 con 0x7f6d64005cb0 2024-09-17T04:33:27.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.873+0000 7f099d72b640 1 Processor -- start 2024-09-17T04:33:27.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.873+0000 7fa97f0f6640 1 Processor -- start 2024-09-17T04:33:27.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.873+0000 7fa97f0f6640 1 -- start start 2024-09-17T04:33:27.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.873+0000 7f1752f07640 1 -- 172.21.15.37:0/1806282560 shutdown_connections 2024-09-17T04:33:27.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.873+0000 7f1752f07640 1 -- 172.21.15.37:0/1806282560 wait complete. 2024-09-17T04:33:27.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.873+0000 7f3184ff9640 1 -- 172.21.15.37:0/1196631736 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f318001a070 con 0x7f31880b2f10 2024-09-17T04:33:27.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.873+0000 7f3184ff9640 1 --2- 172.21.15.37:0/1196631736 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3168078b90 0x7f316807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.872+0000 7fbcd57fa640 1 -- 172.21.15.37:0/2143305204 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbcd80c93f0 con 0x7fbcd0032360 2024-09-17T04:33:27.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.873+0000 7fbcd57fa640 1 --2- 172.21.15.37:0/2143305204 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fbcbc078c90 0x7fbcbc07b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.873+0000 7fbcd57fa640 1 -- 172.21.15.37:0/2143305204 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fbcd8137940 con 0x7fbcd0032360 2024-09-17T04:33:27.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.874+0000 7f3184ff9640 1 -- 172.21.15.37:0/1196631736 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f318009a370 con 0x7f31880b2f10 2024-09-17T04:33:27.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.874+0000 7f099d72b640 1 -- start start 2024-09-17T04:33:27.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.874+0000 7fbcdcebc640 1 --2- 172.21.15.37:0/2143305204 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fbcbc078c90 0x7fbcbc07b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.874+0000 7f3187fff640 1 --2- 172.21.15.37:0/1196631736 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3168078b90 0x7f316807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.874+0000 7f64aff20640 1 Processor -- start 2024-09-17T04:33:27.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.874+0000 7f64aff20640 1 -- start start 2024-09-17T04:33:27.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.875+0000 7f6d537fe640 1 -- 172.21.15.37:0/120885470 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6d6c0bb620 con 0x7f6d64005cb0 2024-09-17T04:33:27.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.875+0000 7fa97f0f6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa978071cd0 0x7fa97813f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.875+0000 7fa97f0f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa978152390 0x7fa978150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.876+0000 7f099d72b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09980b2980 0x7f0998154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.876+0000 7f099d72b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09980cc8b0 0x7f099814a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.876+0000 7f099d72b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f09980cd2b0 0x7f09981f9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.876+0000 7f099d72b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0998162770 con 0x7f09980b2980 2024-09-17T04:33:27.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.876+0000 7f099d72b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f09981625f0 con 0x7f09980cc8b0 2024-09-17T04:33:27.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.876+0000 7fa97e0f4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa978071cd0 0x7fa97813f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.876+0000 7f099d72b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f09981628f0 con 0x7f09980cd2b0 2024-09-17T04:33:27.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.876+0000 7fa97e0f4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa978071cd0 0x7fa97813f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.37:50884/0 (socket says 172.21.15.37:50884) 2024-09-17T04:33:27.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.876+0000 7fa97d8f3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa978152390 0x7fa978150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.876+0000 7fa97d8f3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa978152390 0x7fa978150c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45858/0 (socket says 172.21.15.37:45858) 2024-09-17T04:33:27.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.876+0000 7fa97e0f4640 1 -- 172.21.15.37:0/190591515 learned_addr learned my addr 172.21.15.37:0/190591515 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:27.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.875+0000 7f6d537fe640 1 --2- 172.21.15.37:0/120885470 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6d58078c60 0x7f6d5807b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.876+0000 7f6d537fe640 1 -- 172.21.15.37:0/120885470 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6d6c136ab0 con 0x7f6d64005cb0 2024-09-17T04:33:27.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.876+0000 7fa97f0f6640 1 --2- 172.21.15.37:0/190591515 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa9781fc930 0x7fa9781f92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.876+0000 7fa97f0f6640 1 -- 172.21.15.37:0/190591515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa9780d0600 con 0x7fa978152390 2024-09-17T04:33:27.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.876+0000 7fa97f0f6640 1 -- 172.21.15.37:0/190591515 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa9780d0480 con 0x7fa978071cd0 2024-09-17T04:33:27.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.876+0000 7fa97f0f6640 1 -- 172.21.15.37:0/190591515 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fa9780d0780 con 0x7fa9781fc930 2024-09-17T04:33:27.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.875+0000 7fbcdcebc640 1 --2- 172.21.15.37:0/2143305204 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fbcbc078c90 0x7fbcbc07b1a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbccc001840 tx=0x7fbccc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:27.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.876+0000 7f09977fe640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f09980cd2b0 0x7f09981f9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.876+0000 7f09967fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09980cc8b0 0x7f099814a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.876+0000 7fa97e8f5640 1 --2- 172.21.15.37:0/190591515 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa9781fc930 0x7fa9781f92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.876+0000 7f09967fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09980cc8b0 0x7f099814a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.37:50900/0 (socket says 172.21.15.37:50900) 2024-09-17T04:33:27.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.876+0000 7f09967fc640 1 -- 172.21.15.37:0/2073147742 learned_addr learned my addr 172.21.15.37:0/2073147742 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:27.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.877+0000 7f3184ff9640 1 -- 172.21.15.37:0/1196631736 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3180021020 con 0x7f31880b2f10 2024-09-17T04:33:27.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.877+0000 7f1752f07640 1 Processor -- start 2024-09-17T04:33:27.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.877+0000 7f64aff20640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f64a00e0b30 0x7f64a00e3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.877+0000 7f09977fe640 1 -- 172.21.15.37:0/2073147742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09980cc8b0 msgr2=0x7f099814a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:27.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.877+0000 7f09977fe640 1 --2- 172.21.15.37:0/2073147742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09980cc8b0 0x7f099814a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.877+0000 7f09977fe640 1 -- 172.21.15.37:0/2073147742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09980b2980 msgr2=0x7f0998154710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:27.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.877+0000 7f09977fe640 1 --2- 172.21.15.37:0/2073147742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09980b2980 0x7f0998154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.877+0000 7f09977fe640 1 -- 172.21.15.37:0/2073147742 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0998156c70 con 0x7f09980cd2b0 2024-09-17T04:33:27.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.877+0000 7fa97e0f4640 1 -- 172.21.15.37:0/190591515 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa9781fc930 msgr2=0x7fa9781f92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:27.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.877+0000 7fa97e0f4640 1 --2- 172.21.15.37:0/190591515 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa9781fc930 0x7fa9781f92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.877+0000 7fa97e0f4640 1 -- 172.21.15.37:0/190591515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa978152390 msgr2=0x7fa978150c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:27.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.877+0000 7fa97e0f4640 1 --2- 172.21.15.37:0/190591515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa978152390 0x7fa978150c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.877+0000 7fa97e0f4640 1 -- 172.21.15.37:0/190591515 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9781516f0 con 0x7fa978071cd0 2024-09-17T04:33:27.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.877+0000 7fa97e8f5640 1 --2- 172.21.15.37:0/190591515 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa9781fc930 0x7fa9781f92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:33:27.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.877+0000 7fa97d8f3640 1 --2- 172.21.15.37:0/190591515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa978152390 0x7fa978150c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:33:27.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.877+0000 7f64aff20640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64a00e0f40 0x7f64a00d5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.877+0000 7f64adcbc640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f64a00e0b30 0x7f64a00e3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.877+0000 7f1752f07640 1 -- start start 2024-09-17T04:33:27.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.877+0000 7f3187fff640 1 --2- 172.21.15.37:0/1196631736 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3168078b90 0x7f316807b0a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3178009b30 tx=0x7f3178004ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:27.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.877+0000 7f64adcbc640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f64a00e0b30 0x7f64a00e3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.37:38924/0 (socket says 172.21.15.37:38924) 2024-09-17T04:33:27.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.877+0000 7f09977fe640 1 --2- 172.21.15.37:0/2073147742 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f09980cd2b0 0x7f09981f9370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0990032000 tx=0x7f0990003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:27.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.877+0000 7fa97e0f4640 1 --2- 172.21.15.37:0/190591515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa978071cd0 0x7fa97813f980 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa96800c400 tx=0x7fa96800c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:27.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.877+0000 7f64adcbc640 1 -- 172.21.15.37:0/2331018790 learned_addr learned my addr 172.21.15.37:0/2331018790 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:27.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.877+0000 7f64aff20640 1 --2- 172.21.15.37:0/2331018790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64a00e48f0 0x7f64a00e1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.877+0000 7f64aff20640 1 -- 172.21.15.37:0/2331018790 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f64a0007d40 con 0x7f64a00e0f40 2024-09-17T04:33:27.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.877+0000 7f64aff20640 1 -- 172.21.15.37:0/2331018790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f64a0007bc0 con 0x7f64a00e48f0 2024-09-17T04:33:27.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.877+0000 7f64aff20640 1 -- 172.21.15.37:0/2331018790 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f64a0007ec0 con 0x7f64a00e0b30 2024-09-17T04:33:27.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.878+0000 7f64ad4bb640 1 --2- 172.21.15.37:0/2331018790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64a00e0f40 0x7f64a00d5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.878+0000 7f64ae4bd640 1 --2- 172.21.15.37:0/2331018790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64a00e48f0 0x7f64a00e1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.878+0000 7f6d6a7fc640 1 --2- 172.21.15.37:0/120885470 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6d58078c60 0x7f6d5807b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.878+0000 7fa96effd640 1 -- 172.21.15.37:0/190591515 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968017070 con 0x7fa978071cd0 2024-09-17T04:33:27.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.878+0000 7fa96effd640 1 -- 172.21.15.37:0/190591515 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9680053c0 con 0x7fa978071cd0 2024-09-17T04:33:27.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.878+0000 7fa96effd640 1 -- 172.21.15.37:0/190591515 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968003850 con 0x7fa978071cd0 2024-09-17T04:33:27.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.878+0000 7f64adcbc640 1 -- 172.21.15.37:0/2331018790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64a00e48f0 msgr2=0x7f64a00e1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:27.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.879+0000 7f6d6a7fc640 1 --2- 172.21.15.37:0/120885470 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6d58078c60 0x7f6d5807b170 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6d64170c10 tx=0x7f6d60008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:27.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.878+0000 7f64adcbc640 1 --2- 172.21.15.37:0/2331018790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64a00e48f0 0x7f64a00e1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.878+0000 7f64adcbc640 1 -- 172.21.15.37:0/2331018790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64a00e0f40 msgr2=0x7f64a00d5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:27.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.878+0000 7f64adcbc640 1 --2- 172.21.15.37:0/2331018790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64a00e0f40 0x7f64a00d5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.878+0000 7f64adcbc640 1 -- 172.21.15.37:0/2331018790 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a017d680 con 0x7f64a00e0b30 2024-09-17T04:33:27.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.879+0000 7f64ad4bb640 1 --2- 172.21.15.37:0/2331018790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64a00e0f40 0x7f64a00d5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:33:27.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.878+0000 7fa97f0f6640 1 -- 172.21.15.37:0/190591515 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9781fd4a0 con 0x7fa978071cd0 2024-09-17T04:33:27.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.878+0000 7fa97f0f6640 1 -- 172.21.15.37:0/190591515 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa97813eea0 con 0x7fa978071cd0 2024-09-17T04:33:27.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.879+0000 7f64adcbc640 1 --2- 172.21.15.37:0/2331018790 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f64a00e0b30 0x7f64a00e3f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f649c000c00 tx=0x7f649c0055e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:27.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.880+0000 7fa97f0f6640 1 -- 172.21.15.37:0/190591515 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9780d0c30 con 0x7fa978071cd0 2024-09-17T04:33:27.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.880+0000 7f1752f07640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1744009060 0x7f174417dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.880+0000 7f1752f07640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17440356e0 0x7f174404d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.880+0000 7f1751f05640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1744009060 0x7f174417dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.880+0000 7f1751f05640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1744009060 0x7f174417dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45882/0 (socket says 172.21.15.37:45882) 2024-09-17T04:33:27.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.880+0000 7f1751f05640 1 -- 172.21.15.37:0/3137676160 learned_addr learned my addr 172.21.15.37:0/3137676160 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:27.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.880+0000 7f1752f07640 1 --2- 172.21.15.37:0/3137676160 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1744055130 0x7f17440dd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.880+0000 7f1751704640 1 --2- 172.21.15.37:0/3137676160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17440356e0 0x7f174404d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.880+0000 7f1752f07640 1 -- 172.21.15.37:0/3137676160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1744053d10 con 0x7f1744009060 2024-09-17T04:33:27.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.880+0000 7f1752f07640 1 -- 172.21.15.37:0/3137676160 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1744053b90 con 0x7f17440356e0 2024-09-17T04:33:27.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.880+0000 7f1752706640 1 --2- 172.21.15.37:0/3137676160 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1744055130 0x7f17440dd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.880+0000 7f1752f07640 1 -- 172.21.15.37:0/3137676160 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f1744053e90 con 0x7f1744055130 2024-09-17T04:33:27.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.881+0000 7fa96effd640 1 -- 172.21.15.37:0/190591515 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa96801a070 con 0x7fa978071cd0 2024-09-17T04:33:27.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.881+0000 7f1751704640 1 -- 172.21.15.37:0/3137676160 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1744055130 msgr2=0x7f17440dd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:27.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.881+0000 7f1751704640 1 --2- 172.21.15.37:0/3137676160 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1744055130 0x7f17440dd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.881+0000 7f1751704640 1 -- 172.21.15.37:0/3137676160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1744009060 msgr2=0x7f174417dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:27.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.881+0000 7f1751704640 1 --2- 172.21.15.37:0/3137676160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1744009060 0x7f174417dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:27.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.881+0000 7f1751704640 1 -- 172.21.15.37:0/3137676160 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174417d270 con 0x7f17440356e0 2024-09-17T04:33:27.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.881+0000 7f649a7fc640 1 -- 172.21.15.37:0/2331018790 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c03f020 con 0x7f64a00e0b30 2024-09-17T04:33:27.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.881+0000 7f1751704640 1 --2- 172.21.15.37:0/3137676160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17440356e0 0x7f174404d180 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f173c00e9f0 tx=0x7f173c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:27.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.881+0000 7f649a7fc640 1 -- 172.21.15.37:0/2331018790 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f649c004440 con 0x7f64a00e0b30 2024-09-17T04:33:27.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.881+0000 7f097f7fe640 1 -- 172.21.15.37:0/2073147742 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0990038040 con 0x7f09980cd2b0 2024-09-17T04:33:27.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.881+0000 7f097f7fe640 1 -- 172.21.15.37:0/2073147742 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0990041070 con 0x7f09980cd2b0 2024-09-17T04:33:27.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.881+0000 7f097f7fe640 1 -- 172.21.15.37:0/2073147742 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0990004590 con 0x7f09980cd2b0 2024-09-17T04:33:27.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.881+0000 7f099d72b640 1 -- 172.21.15.37:0/2073147742 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0998156f50 con 0x7f09980cd2b0 2024-09-17T04:33:27.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.882+0000 7f099d72b640 1 -- 172.21.15.37:0/2073147742 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f09980ca610 con 0x7f09980cd2b0 2024-09-17T04:33:27.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.881+0000 7f649a7fc640 1 -- 172.21.15.37:0/2331018790 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c004a00 con 0x7f64a00e0b30 2024-09-17T04:33:27.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.882+0000 7f64aff20640 1 -- 172.21.15.37:0/2331018790 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a017d960 con 0x7f64a00e0b30 2024-09-17T04:33:27.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.882+0000 7f64aff20640 1 -- 172.21.15.37:0/2331018790 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a017df20 con 0x7f64a00e0b30 2024-09-17T04:33:27.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.882+0000 7fa96effd640 1 --2- 172.21.15.37:0/190591515 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa95c078c10 0x7fa95c07b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.882+0000 7fa96effd640 1 -- 172.21.15.37:0/190591515 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa968099e60 con 0x7fa978071cd0 2024-09-17T04:33:27.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.883+0000 7fa97d8f3640 1 --2- 172.21.15.37:0/190591515 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa95c078c10 0x7fa95c07b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.881+0000 7fbcd57fa640 1 -- 172.21.15.37:0/2143305204 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fbcd80ffe80 con 0x7fbcd0032360 2024-09-17T04:33:27.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.883+0000 7f64aff20640 1 -- 172.21.15.37:0/2331018790 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64a0052f20 con 0x7f64a00e0b30 2024-09-17T04:33:27.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.884+0000 7f6d537fe640 1 -- 172.21.15.37:0/120885470 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6d6402aec0 con 0x7f6d64005cb0 2024-09-17T04:33:27.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.884+0000 7fa97d8f3640 1 --2- 172.21.15.37:0/190591515 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa95c078c10 0x7fa95c07b120 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa97400b3c0 tx=0x7fa97400d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:27.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.884+0000 7f097f7fe640 1 -- 172.21.15.37:0/2073147742 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0990046460 con 0x7f09980cd2b0 2024-09-17T04:33:27.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.884+0000 7f099d72b640 1 -- 172.21.15.37:0/2073147742 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0998161ea0 con 0x7f09980cd2b0 2024-09-17T04:33:27.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.885+0000 7f097f7fe640 1 --2- 172.21.15.37:0/2073147742 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f0984078c90 0x7f098407b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.885+0000 7f649a7fc640 1 -- 172.21.15.37:0/2331018790 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f649c042020 con 0x7f64a00e0b30 2024-09-17T04:33:27.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.885+0000 7f097f7fe640 1 -- 172.21.15.37:0/2073147742 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f09900bb590 con 0x7f09980cd2b0 2024-09-17T04:33:27.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.885+0000 7f0996ffd640 1 --2- 172.21.15.37:0/2073147742 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f0984078c90 0x7f098407b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.886+0000 7f0996ffd640 1 --2- 172.21.15.37:0/2073147742 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f0984078c90 0x7f098407b1a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f09980d01f0 tx=0x7f0988007a50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:27.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.886+0000 7f649a7fc640 1 --2- 172.21.15.37:0/2331018790 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f648c078bc0 0x7f648c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.886+0000 7f649a7fc640 1 -- 172.21.15.37:0/2331018790 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f649c0bbc60 con 0x7f64a00e0b30 2024-09-17T04:33:27.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.886+0000 7f64ad4bb640 1 --2- 172.21.15.37:0/2331018790 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f648c078bc0 0x7f648c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.887+0000 7f64ad4bb640 1 --2- 172.21.15.37:0/2331018790 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f648c078bc0 0x7f648c07b0d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f64a00de520 tx=0x7f64a400b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:27.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.888+0000 7fa96effd640 1 -- 172.21.15.37:0/190591515 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa968061e50 con 0x7fa978071cd0 2024-09-17T04:33:27.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.888+0000 7f17427fc640 1 -- 172.21.15.37:0/3137676160 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c00ccc0 con 0x7f17440356e0 2024-09-17T04:33:27.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.888+0000 7f17427fc640 1 -- 172.21.15.37:0/3137676160 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f173c010430 con 0x7f17440356e0 2024-09-17T04:33:27.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.888+0000 7f17427fc640 1 -- 172.21.15.37:0/3137676160 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c010860 con 0x7f17440356e0 2024-09-17T04:33:27.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.889+0000 7f1752f07640 1 -- 172.21.15.37:0/3137676160 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f174404dad0 con 0x7f17440356e0 2024-09-17T04:33:27.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.889+0000 7f1752f07640 1 -- 172.21.15.37:0/3137676160 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f174417d7c0 con 0x7f17440356e0 2024-09-17T04:33:27.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.890+0000 7f1752f07640 1 -- 172.21.15.37:0/3137676160 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1744053440 con 0x7f17440356e0 2024-09-17T04:33:27.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.892+0000 7f17427fc640 1 -- 172.21.15.37:0/3137676160 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f173c010a00 con 0x7f17440356e0 2024-09-17T04:33:27.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.892+0000 7f097f7fe640 1 -- 172.21.15.37:0/2073147742 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0990083ad0 con 0x7f09980cd2b0 2024-09-17T04:33:27.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.892+0000 7f17427fc640 1 --2- 172.21.15.37:0/3137676160 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1730078be0 0x7f173007b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:27.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.893+0000 7f17427fc640 1 -- 172.21.15.37:0/3137676160 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f173c09b7b0 con 0x7f17440356e0 2024-09-17T04:33:27.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.893+0000 7f1751f05640 1 --2- 172.21.15.37:0/3137676160 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1730078be0 0x7f173007b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:27.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.893+0000 7f1751f05640 1 --2- 172.21.15.37:0/3137676160 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1730078be0 0x7f173007b0f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1748004f70 tx=0x7f1748007ef0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:27.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.902+0000 7f649a7fc640 1 -- 172.21.15.37:0/2331018790 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f649c049030 con 0x7f64a00e0b30 2024-09-17T04:33:27.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:27.902+0000 7f17427fc640 1 -- 172.21.15.37:0/3137676160 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f173c063610 con 0x7f17440356e0 2024-09-17T04:33:28.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.225+0000 7f6f75e8b640 1 -- 172.21.15.37:0/4216776464 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f6f50005580 con 0x7f6f70159fb0 2024-09-17T04:33:28.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.225+0000 7f6f57fff640 1 -- 172.21.15.37:0/4216776464 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6f60067890 con 0x7f6f70159fb0 2024-09-17T04:33:28.225 INFO:teuthology.orchestra.run.smithi037.stdout:60129542150 2024-09-17T04:33:28.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.229+0000 7f6f55ffb640 1 -- 172.21.15.37:0/4216776464 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6f58078d80 msgr2=0x7f6f5807b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.229+0000 7f6f55ffb640 1 --2- 172.21.15.37:0/4216776464 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6f58078d80 0x7f6f5807b290 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6f680012e0 tx=0x7f6f68009040 comp rx=0 tx=0).stop 2024-09-17T04:33:28.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.230+0000 7f6f55ffb640 1 -- 172.21.15.37:0/4216776464 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6f70159fb0 msgr2=0x7f6f701567f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.230+0000 7f6f55ffb640 1 --2- 172.21.15.37:0/4216776464 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6f70159fb0 0x7f6f701567f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000e720 tx=0x7f6f6000ebe0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.230+0000 7f6f55ffb640 1 -- 172.21.15.37:0/4216776464 shutdown_connections 2024-09-17T04:33:28.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.230+0000 7f6f55ffb640 1 --2- 172.21.15.37:0/4216776464 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6f58078d80 0x7f6f5807b290 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.230+0000 7f6f55ffb640 1 --2- 172.21.15.37:0/4216776464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f7015a900 0x7f6f7009bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.230+0000 7f6f55ffb640 1 --2- 172.21.15.37:0/4216776464 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6f70159fb0 0x7f6f701567f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.230+0000 7f6f55ffb640 1 --2- 172.21.15.37:0/4216776464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f700d0bf0 0x7f6f700c4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.230+0000 7f6f55ffb640 1 -- 172.21.15.37:0/4216776464 >> 172.21.15.37:0/4216776464 conn(0x7f6f700d00d0 msgr2=0x7f6f700ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:28.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.231+0000 7f6f55ffb640 1 -- 172.21.15.37:0/4216776464 shutdown_connections 2024-09-17T04:33:28.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.231+0000 7f6f55ffb640 1 -- 172.21.15.37:0/4216776464 wait complete. 2024-09-17T04:33:28.247 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.1 2024-09-17T04:33:28.247 DEBUG:teuthology.parallel:result is None 2024-09-17T04:33:28.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.275+0000 7fa97f0f6640 1 -- 172.21.15.37:0/190591515 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fa9780b2990 con 0x7fa978071cd0 2024-09-17T04:33:28.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.275+0000 7fa96effd640 1 -- 172.21.15.37:0/190591515 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa968066880 con 0x7fa978071cd0 2024-09-17T04:33:28.276 INFO:teuthology.orchestra.run.smithi037.stdout:60129542151 2024-09-17T04:33:28.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.278+0000 7fa97f0f6640 1 -- 172.21.15.37:0/190591515 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa95c078c10 msgr2=0x7fa95c07b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.278+0000 7fa97f0f6640 1 --2- 172.21.15.37:0/190591515 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa95c078c10 0x7fa95c07b120 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa97400b3c0 tx=0x7fa97400d040 comp rx=0 tx=0).stop 2024-09-17T04:33:28.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.278+0000 7fa97f0f6640 1 -- 172.21.15.37:0/190591515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa978071cd0 msgr2=0x7fa97813f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.278+0000 7fa97f0f6640 1 --2- 172.21.15.37:0/190591515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa978071cd0 0x7fa97813f980 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa96800c400 tx=0x7fa96800c8c0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.278+0000 7fa97f0f6640 1 -- 172.21.15.37:0/190591515 shutdown_connections 2024-09-17T04:33:28.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.278+0000 7fa97f0f6640 1 --2- 172.21.15.37:0/190591515 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa95c078c10 0x7fa95c07b120 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.278+0000 7fa97f0f6640 1 --2- 172.21.15.37:0/190591515 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa9781fc930 0x7fa9781f92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.278+0000 7fa97f0f6640 1 --2- 172.21.15.37:0/190591515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa978152390 0x7fa978150c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.278+0000 7fa97f0f6640 1 --2- 172.21.15.37:0/190591515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa978071cd0 0x7fa97813f980 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.278+0000 7fa97f0f6640 1 -- 172.21.15.37:0/190591515 >> 172.21.15.37:0/190591515 conn(0x7fa9780b2bb0 msgr2=0x7fa9780b5ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:28.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.280+0000 7fa97f0f6640 1 -- 172.21.15.37:0/190591515 shutdown_connections 2024-09-17T04:33:28.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.280+0000 7fa97f0f6640 1 -- 172.21.15.37:0/190591515 wait complete. 2024-09-17T04:33:28.291 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.0 2024-09-17T04:33:28.291 DEBUG:teuthology.parallel:result is None 2024-09-17T04:33:28.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.305+0000 7f6d517fa640 1 -- 172.21.15.37:0/120885470 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f6d640df180 con 0x7f6d64005cb0 2024-09-17T04:33:28.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.305+0000 7f6d537fe640 1 -- 172.21.15.37:0/120885470 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6d6418aa50 con 0x7f6d64005cb0 2024-09-17T04:33:28.305 INFO:teuthology.orchestra.run.smithi037.stdout:60129542151 2024-09-17T04:33:28.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.306+0000 7f318d358640 1 -- 172.21.15.37:0/1196631736 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f318806dd70 con 0x7f31880b2f10 2024-09-17T04:33:28.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.306+0000 7f3184ff9640 1 -- 172.21.15.37:0/1196631736 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3180062290 con 0x7f31880b2f10 2024-09-17T04:33:28.307 INFO:teuthology.orchestra.run.smithi037.stdout:60129542151 2024-09-17T04:33:28.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.309+0000 7f318d358640 1 -- 172.21.15.37:0/1196631736 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3168078b90 msgr2=0x7f316807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.309+0000 7f318d358640 1 --2- 172.21.15.37:0/1196631736 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3168078b90 0x7f316807b0a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3178009b30 tx=0x7f3178004ab0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.309+0000 7f6d517fa640 1 -- 172.21.15.37:0/120885470 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6d58078c60 msgr2=0x7f6d5807b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.309+0000 7f6d517fa640 1 --2- 172.21.15.37:0/120885470 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6d58078c60 0x7f6d5807b170 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6d64170c10 tx=0x7f6d60008040 comp rx=0 tx=0).stop 2024-09-17T04:33:28.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.309+0000 7f6d517fa640 1 -- 172.21.15.37:0/120885470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d64005cb0 msgr2=0x7f6d640de580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.309+0000 7f318d358640 1 -- 172.21.15.37:0/1196631736 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f31880b2f10 msgr2=0x7f31880cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.309+0000 7f318d358640 1 --2- 172.21.15.37:0/1196631736 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f31880b2f10 0x7f31880cfd50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f318000c390 tx=0x7f318000c850 comp rx=0 tx=0).stop 2024-09-17T04:33:28.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.309+0000 7f318d358640 1 -- 172.21.15.37:0/1196631736 shutdown_connections 2024-09-17T04:33:28.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.309+0000 7f318d358640 1 --2- 172.21.15.37:0/1196631736 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3168078b90 0x7f316807b0a0 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3178009b30 tx=0x7f3178004ab0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.309+0000 7f318d358640 1 --2- 172.21.15.37:0/1196631736 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f31880b2f10 0x7f31880cfd50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.309+0000 7f318d358640 1 --2- 172.21.15.37:0/1196631736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3188073710 0x7f31880adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.309+0000 7f318d358640 1 --2- 172.21.15.37:0/1196631736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3188070a80 0x7f3188209f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.309+0000 7f318d358640 1 -- 172.21.15.37:0/1196631736 >> 172.21.15.37:0/1196631736 conn(0x7f31880b2000 msgr2=0x7f318815df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:28.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.309+0000 7f318d358640 1 -- 172.21.15.37:0/1196631736 shutdown_connections 2024-09-17T04:33:28.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.309+0000 7f6d517fa640 1 --2- 172.21.15.37:0/120885470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d64005cb0 0x7f6d640de580 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c077560 tx=0x7f6d6c0b1680 comp rx=0 tx=0).stop 2024-09-17T04:33:28.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.309+0000 7f6d517fa640 1 -- 172.21.15.37:0/120885470 shutdown_connections 2024-09-17T04:33:28.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.309+0000 7f6d517fa640 1 --2- 172.21.15.37:0/120885470 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6d58078c60 0x7f6d5807b170 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.309+0000 7f6d517fa640 1 --2- 172.21.15.37:0/120885470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d64021a50 0x7f6d64021e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.309+0000 7f6d517fa640 1 --2- 172.21.15.37:0/120885470 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6d6418a100 0x7f6d6418a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.309+0000 7f6d517fa640 1 --2- 172.21.15.37:0/120885470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d64005cb0 0x7f6d640de580 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.309+0000 7f6d517fa640 1 -- 172.21.15.37:0/120885470 >> 172.21.15.37:0/120885470 conn(0x7f6d64051420 msgr2=0x7f6d6402a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:28.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.309+0000 7f318d358640 1 -- 172.21.15.37:0/1196631736 wait complete. 2024-09-17T04:33:28.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.310+0000 7f6d517fa640 1 -- 172.21.15.37:0/120885470 shutdown_connections 2024-09-17T04:33:28.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.310+0000 7f6d517fa640 1 -- 172.21.15.37:0/120885470 wait complete. 2024-09-17T04:33:28.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.318+0000 7fbcc2ffd640 1 -- 172.21.15.37:0/2143305204 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fbcd0055710 con 0x7fbcd0032360 2024-09-17T04:33:28.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.318+0000 7fbcd57fa640 1 -- 172.21.15.37:0/2143305204 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbcd80ff780 con 0x7fbcd0032360 2024-09-17T04:33:28.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.318+0000 7f099d72b640 1 -- 172.21.15.37:0/2073147742 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f09980b7120 con 0x7f09980cd2b0 2024-09-17T04:33:28.318 INFO:teuthology.orchestra.run.smithi037.stdout:60129542151 2024-09-17T04:33:28.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.319+0000 7f097f7fe640 1 -- 172.21.15.37:0/2073147742 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f09900833d0 con 0x7f09980cd2b0 2024-09-17T04:33:28.319 INFO:teuthology.orchestra.run.smithi037.stdout:60129542152 2024-09-17T04:33:28.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.322+0000 7fbcc2ffd640 1 -- 172.21.15.37:0/2143305204 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fbcbc078c90 msgr2=0x7fbcbc07b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.322+0000 7fbcc2ffd640 1 --2- 172.21.15.37:0/2143305204 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fbcbc078c90 0x7fbcbc07b1a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbccc001840 tx=0x7fbccc009040 comp rx=0 tx=0).stop 2024-09-17T04:33:28.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.322+0000 7fbcc2ffd640 1 -- 172.21.15.37:0/2143305204 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbcd0032360 msgr2=0x7fbcd0033200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.322+0000 7fbcc2ffd640 1 --2- 172.21.15.37:0/2143305204 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbcd0032360 0x7fbcd0033200 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8051d10 tx=0x7fbcd80adcc0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.322+0000 7fbcc2ffd640 1 -- 172.21.15.37:0/2143305204 shutdown_connections 2024-09-17T04:33:28.324 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.4 2024-09-17T04:33:28.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.322+0000 7f099d72b640 1 -- 172.21.15.37:0/2073147742 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f0984078c90 msgr2=0x7f098407b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.322+0000 7f099d72b640 1 --2- 172.21.15.37:0/2073147742 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f0984078c90 0x7f098407b1a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f09980d01f0 tx=0x7f0988007a50 comp rx=0 tx=0).stop 2024-09-17T04:33:28.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.322+0000 7f099d72b640 1 -- 172.21.15.37:0/2073147742 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f09980cd2b0 msgr2=0x7f09981f9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.322+0000 7f099d72b640 1 --2- 172.21.15.37:0/2073147742 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f09980cd2b0 0x7f09981f9370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0990032000 tx=0x7f0990003f10 comp rx=0 tx=0).stop 2024-09-17T04:33:28.325 DEBUG:teuthology.parallel:result is None 2024-09-17T04:33:28.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.322+0000 7fbcc2ffd640 1 --2- 172.21.15.37:0/2143305204 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fbcbc078c90 0x7fbcbc07b1a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.322+0000 7fbcc2ffd640 1 --2- 172.21.15.37:0/2143305204 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbcd0032360 0x7fbcd0033200 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.322+0000 7fbcc2ffd640 1 --2- 172.21.15.37:0/2143305204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbcd0007680 0x7fbcd0032cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.322+0000 7fbcc2ffd640 1 --2- 172.21.15.37:0/2143305204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcd0004a70 0x7fbcd00d0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.322+0000 7fbcc2ffd640 1 -- 172.21.15.37:0/2143305204 >> 172.21.15.37:0/2143305204 conn(0x7fbcd001ddf0 msgr2=0x7fbcd00363f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:28.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.322+0000 7f099d72b640 1 -- 172.21.15.37:0/2073147742 shutdown_connections 2024-09-17T04:33:28.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.322+0000 7f099d72b640 1 --2- 172.21.15.37:0/2073147742 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f0984078c90 0x7f098407b1a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.322+0000 7f099d72b640 1 --2- 172.21.15.37:0/2073147742 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f09980cd2b0 0x7f09981f9370 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.322+0000 7f099d72b640 1 --2- 172.21.15.37:0/2073147742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09980cc8b0 0x7f099814a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.322+0000 7f099d72b640 1 --2- 172.21.15.37:0/2073147742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09980b2980 0x7f0998154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.322+0000 7f099d72b640 1 -- 172.21.15.37:0/2073147742 >> 172.21.15.37:0/2073147742 conn(0x7f099809aa20 msgr2=0x7f09980cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:28.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.322+0000 7f099d72b640 1 -- 172.21.15.37:0/2073147742 shutdown_connections 2024-09-17T04:33:28.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.322+0000 7f099d72b640 1 -- 172.21.15.37:0/2073147742 wait complete. 2024-09-17T04:33:28.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.322+0000 7fbcc2ffd640 1 -- 172.21.15.37:0/2143305204 shutdown_connections 2024-09-17T04:33:28.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.323+0000 7fbcc2ffd640 1 -- 172.21.15.37:0/2143305204 wait complete. 2024-09-17T04:33:28.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.323+0000 7f1752f07640 1 -- 172.21.15.37:0/3137676160 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f1744052ef0 con 0x7f17440356e0 2024-09-17T04:33:28.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.324+0000 7f17427fc640 1 -- 172.21.15.37:0/3137676160 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f173c068040 con 0x7f17440356e0 2024-09-17T04:33:28.328 INFO:teuthology.orchestra.run.smithi037.stdout:60129542152 2024-09-17T04:33:28.329 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.6 2024-09-17T04:33:28.329 DEBUG:teuthology.parallel:result is None 2024-09-17T04:33:28.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.327+0000 7f1752f07640 1 -- 172.21.15.37:0/3137676160 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1730078be0 msgr2=0x7f173007b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.327+0000 7f1752f07640 1 --2- 172.21.15.37:0/3137676160 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1730078be0 0x7f173007b0f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1748004f70 tx=0x7f1748007ef0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.328+0000 7f1752f07640 1 -- 172.21.15.37:0/3137676160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17440356e0 msgr2=0x7f174404d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.328+0000 7f1752f07640 1 --2- 172.21.15.37:0/3137676160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17440356e0 0x7f174404d180 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f173c00e9f0 tx=0x7f173c00eeb0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.328+0000 7f1752f07640 1 -- 172.21.15.37:0/3137676160 shutdown_connections 2024-09-17T04:33:28.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.328+0000 7f1752f07640 1 --2- 172.21.15.37:0/3137676160 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1730078be0 0x7f173007b0f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.328+0000 7f1752f07640 1 --2- 172.21.15.37:0/3137676160 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1744055130 0x7f17440dd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.328+0000 7f1752f07640 1 --2- 172.21.15.37:0/3137676160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17440356e0 0x7f174404d180 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.328+0000 7f1752f07640 1 --2- 172.21.15.37:0/3137676160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1744009060 0x7f174417dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.328+0000 7f1752f07640 1 -- 172.21.15.37:0/3137676160 >> 172.21.15.37:0/3137676160 conn(0x7f174401eac0 msgr2=0x7f1744007dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:28.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.328+0000 7f1752f07640 1 -- 172.21.15.37:0/3137676160 shutdown_connections 2024-09-17T04:33:28.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.328+0000 7f1752f07640 1 -- 172.21.15.37:0/3137676160 wait complete. 2024-09-17T04:33:28.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.329+0000 7f64aff20640 1 -- 172.21.15.37:0/2331018790 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f64a00e0300 con 0x7f64a00e0b30 2024-09-17T04:33:28.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.330+0000 7f649a7fc640 1 -- 172.21.15.37:0/2331018790 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f64a0054e70 con 0x7f64a00e0b30 2024-09-17T04:33:28.332 INFO:teuthology.orchestra.run.smithi037.stdout:60129542151 2024-09-17T04:33:28.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.333+0000 7f64aff20640 1 -- 172.21.15.37:0/2331018790 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f648c078bc0 msgr2=0x7f648c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.333+0000 7f64aff20640 1 --2- 172.21.15.37:0/2331018790 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f648c078bc0 0x7f648c07b0d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f64a00de520 tx=0x7f64a400b2e0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.333+0000 7f64aff20640 1 -- 172.21.15.37:0/2331018790 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f64a00e0b30 msgr2=0x7f64a00e3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.333+0000 7f64aff20640 1 --2- 172.21.15.37:0/2331018790 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f64a00e0b30 0x7f64a00e3f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f649c000c00 tx=0x7f649c0055e0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.333+0000 7f64aff20640 1 -- 172.21.15.37:0/2331018790 shutdown_connections 2024-09-17T04:33:28.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.333+0000 7f64aff20640 1 --2- 172.21.15.37:0/2331018790 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f648c078bc0 0x7f648c07b0d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.333+0000 7f64aff20640 1 --2- 172.21.15.37:0/2331018790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64a00e48f0 0x7f64a00e1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.333+0000 7f64aff20640 1 --2- 172.21.15.37:0/2331018790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64a00e0f40 0x7f64a00d5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.333+0000 7f64aff20640 1 --2- 172.21.15.37:0/2331018790 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f64a00e0b30 0x7f64a00e3f20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.333+0000 7f64aff20640 1 -- 172.21.15.37:0/2331018790 >> 172.21.15.37:0/2331018790 conn(0x7f64a001eaf0 msgr2=0x7f64a00df1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:28.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.333+0000 7f64aff20640 1 -- 172.21.15.37:0/2331018790 shutdown_connections 2024-09-17T04:33:28.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.333+0000 7f64aff20640 1 -- 172.21.15.37:0/2331018790 wait complete. 2024-09-17T04:33:28.337 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.5 2024-09-17T04:33:28.337 DEBUG:teuthology.parallel:result is None 2024-09-17T04:33:28.342 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.7 2024-09-17T04:33:28.342 DEBUG:teuthology.parallel:result is None 2024-09-17T04:33:28.346 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.3 2024-09-17T04:33:28.346 DEBUG:teuthology.parallel:result is None 2024-09-17T04:33:28.346 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.2 2024-09-17T04:33:28.347 DEBUG:teuthology.parallel:result is None 2024-09-17T04:33:28.347 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T04:33:28.347 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:33:28.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.502+0000 7f657c358640 1 -- 172.21.15.37:0/1357420236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f657415c830 msgr2=0x7f65740d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.502+0000 7f657c358640 1 --2- 172.21.15.37:0/1357420236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f657415c830 0x7f65740d0240 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f65740b1710 tx=0x7f656402aa30 comp rx=0 tx=0).stop 2024-09-17T04:33:28.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.502+0000 7f657c358640 1 -- 172.21.15.37:0/1357420236 shutdown_connections 2024-09-17T04:33:28.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.502+0000 7f657c358640 1 --2- 172.21.15.37:0/1357420236 >> v2:172.21.15.92:3300/0 conn(0x7f65740cd090 0x7f65740727f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.502+0000 7f657c358640 1 --2- 172.21.15.37:0/1357420236 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f65740d1a10 0x7f65740ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.502+0000 7f657c358640 1 --2- 172.21.15.37:0/1357420236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f657415c830 0x7f65740d0240 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.502+0000 7f657c358640 1 -- 172.21.15.37:0/1357420236 >> 172.21.15.37:0/1357420236 conn(0x7f6574071050 msgr2=0x7f6574071460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:28.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.502+0000 7f657c358640 1 -- 172.21.15.37:0/1357420236 shutdown_connections 2024-09-17T04:33:28.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.502+0000 7f657c358640 1 -- 172.21.15.37:0/1357420236 wait complete. 2024-09-17T04:33:28.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.504+0000 7f657c358640 1 Processor -- start 2024-09-17T04:33:28.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.504+0000 7f657c358640 1 -- start start 2024-09-17T04:33:28.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.505+0000 7f657c358640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65740cd090 0x7f6574154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:28.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.505+0000 7f657c358640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f65740d1a10 0x7f6574156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:28.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.505+0000 7f657c358640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f657415c830 0x7f65741f3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:28.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.505+0000 7f657c358640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f65740730c0 con 0x7f65740cd090 2024-09-17T04:33:28.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.505+0000 7f657c358640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6574072f40 con 0x7f657415c830 2024-09-17T04:33:28.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.505+0000 7f657c358640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f6574073240 con 0x7f65740d1a10 2024-09-17T04:33:28.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.506+0000 7f657a0f4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65740cd090 0x7f6574154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:28.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.506+0000 7f657a0f4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65740cd090 0x7f6574154d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45904/0 (socket says 172.21.15.37:45904) 2024-09-17T04:33:28.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.506+0000 7f657a0f4640 1 -- 172.21.15.37:0/45596923 learned_addr learned my addr 172.21.15.37:0/45596923 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:28.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.506+0000 7f657a8f5640 1 --2- 172.21.15.37:0/45596923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f657415c830 0x7f65741f3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:28.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.506+0000 7f65798f3640 1 --2- 172.21.15.37:0/45596923 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f65740d1a10 0x7f6574156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:28.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.506+0000 7f657a0f4640 1 -- 172.21.15.37:0/45596923 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f65740d1a10 msgr2=0x7f6574156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.506+0000 7f657a0f4640 1 --2- 172.21.15.37:0/45596923 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f65740d1a10 0x7f6574156f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.506+0000 7f657a0f4640 1 -- 172.21.15.37:0/45596923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f657415c830 msgr2=0x7f65741f3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.506+0000 7f657a0f4640 1 --2- 172.21.15.37:0/45596923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f657415c830 0x7f65741f3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.506+0000 7f657a0f4640 1 -- 172.21.15.37:0/45596923 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65741f4320 con 0x7f65740cd090 2024-09-17T04:33:28.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.506+0000 7f657a0f4640 1 --2- 172.21.15.37:0/45596923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65740cd090 0x7f6574154d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6564009bf0 tx=0x7f6564003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:28.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.506+0000 7f6572ffd640 1 -- 172.21.15.37:0/45596923 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6564033050 con 0x7f65740cd090 2024-09-17T04:33:28.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.506+0000 7f6572ffd640 1 -- 172.21.15.37:0/45596923 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65640044d0 con 0x7f65740cd090 2024-09-17T04:33:28.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.506+0000 7f6572ffd640 1 -- 172.21.15.37:0/45596923 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6564038450 con 0x7f65740cd090 2024-09-17T04:33:28.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.506+0000 7f657c358640 1 -- 172.21.15.37:0/45596923 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6574157480 con 0x7f65740cd090 2024-09-17T04:33:28.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.506+0000 7f657c358640 1 -- 172.21.15.37:0/45596923 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6574154260 con 0x7f65740cd090 2024-09-17T04:33:28.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.507+0000 7f657c358640 1 -- 172.21.15.37:0/45596923 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65740ccb50 con 0x7f65740cd090 2024-09-17T04:33:28.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.508+0000 7f6572ffd640 1 -- 172.21.15.37:0/45596923 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6564036070 con 0x7f65740cd090 2024-09-17T04:33:28.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.509+0000 7f6572ffd640 1 --2- 172.21.15.37:0/45596923 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6560078bc0 0x7f656007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:28.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.509+0000 7f6572ffd640 1 -- 172.21.15.37:0/45596923 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f65640bc180 con 0x7f65740cd090 2024-09-17T04:33:28.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.509+0000 7f65798f3640 1 --2- 172.21.15.37:0/45596923 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6560078bc0 0x7f656007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:28.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.509+0000 7f65798f3640 1 --2- 172.21.15.37:0/45596923 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6560078bc0 0x7f656007b0d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f65740b17b0 tx=0x7f6568005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:28.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.513+0000 7f6572ffd640 1 -- 172.21.15.37:0/45596923 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6564084000 con 0x7f65740cd090 2024-09-17T04:33:28.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.697+0000 7f657c358640 1 -- 172.21.15.37:0/45596923 --> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f65740b5fb0 con 0x7f6560078bc0 2024-09-17T04:33:28.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.702+0000 7f6572ffd640 1 -- 172.21.15.37:0/45596923 <== mgr.4100 v2:172.21.15.92:6800/864337835 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251825 (secure 0 0 0) 0x7f65740b5fb0 con 0x7f6560078bc0 2024-09-17T04:33:28.705 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:28.706 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-17T04:33:28.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.705+0000 7f657c358640 1 -- 172.21.15.37:0/45596923 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6560078bc0 msgr2=0x7f656007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.705+0000 7f657c358640 1 --2- 172.21.15.37:0/45596923 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6560078bc0 0x7f656007b0d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f65740b17b0 tx=0x7f6568005f90 comp rx=0 tx=0).stop 2024-09-17T04:33:28.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.705+0000 7f657c358640 1 -- 172.21.15.37:0/45596923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65740cd090 msgr2=0x7f6574154d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.705+0000 7f657c358640 1 --2- 172.21.15.37:0/45596923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65740cd090 0x7f6574154d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6564009bf0 tx=0x7f6564003f90 comp rx=0 tx=0).stop 2024-09-17T04:33:28.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.705+0000 7f657c358640 1 -- 172.21.15.37:0/45596923 shutdown_connections 2024-09-17T04:33:28.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.705+0000 7f657c358640 1 --2- 172.21.15.37:0/45596923 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6560078bc0 0x7f656007b0d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.705+0000 7f657c358640 1 --2- 172.21.15.37:0/45596923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f657415c830 0x7f65741f3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.705+0000 7f657c358640 1 --2- 172.21.15.37:0/45596923 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f65740d1a10 0x7f6574156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.705+0000 7f657c358640 1 --2- 172.21.15.37:0/45596923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65740cd090 0x7f6574154d40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.705+0000 7f657c358640 1 -- 172.21.15.37:0/45596923 >> 172.21.15.37:0/45596923 conn(0x7f6574071050 msgr2=0x7f657415f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:28.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.705+0000 7f657c358640 1 -- 172.21.15.37:0/45596923 shutdown_connections 2024-09-17T04:33:28.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.705+0000 7f657c358640 1 -- 172.21.15.37:0/45596923 wait complete. 2024-09-17T04:33:28.716 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-17T04:33:28.447609+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219216,"kb_used_data":2984,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755504,"statfs":{"total":773094113280,"available":772869636096,"internally_reserved":0,"allocated":3055616,"data_stored":1816820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.777989"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691577+0000","last_change":"2024-09-17T04:33:19.693609+0000","last_active":"2024-09-17T04:33:20.691577+0000","last_peered":"2024-09-17T04:33:20.691577+0000","last_clean":"2024-09-17T04:33:20.691577+0000","last_became_active":"2024-09-17T04:33:19.693159+0000","last_became_peered":"2024-09-17T04:33:19.693159+0000","last_unstale":"2024-09-17T04:33:20.691577+0000","last_undegraded":"2024-09-17T04:33:20.691577+0000","last_fullsized":"2024-09-17T04:33:20.691577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:29.058321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674309+0000","last_change":"2024-09-17T04:33:19.686724+0000","last_active":"2024-09-17T04:33:21.674309+0000","last_peered":"2024-09-17T04:33:21.674309+0000","last_clean":"2024-09-17T04:33:21.674309+0000","last_became_active":"2024-09-17T04:33:19.685840+0000","last_became_peered":"2024-09-17T04:33:19.685840+0000","last_unstale":"2024-09-17T04:33:21.674309+0000","last_undegraded":"2024-09-17T04:33:21.674309+0000","last_fullsized":"2024-09-17T04:33:21.674309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:21.722762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035198+0000","last_change":"2024-09-17T04:33:19.688324+0000","last_active":"2024-09-17T04:33:22.035198+0000","last_peered":"2024-09-17T04:33:22.035198+0000","last_clean":"2024-09-17T04:33:22.035198+0000","last_became_active":"2024-09-17T04:33:19.687914+0000","last_became_peered":"2024-09-17T04:33:19.687914+0000","last_unstale":"2024-09-17T04:33:22.035198+0000","last_undegraded":"2024-09-17T04:33:22.035198+0000","last_fullsized":"2024-09-17T04:33:22.035198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:44.753208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930207+0000","last_change":"2024-09-17T04:33:19.694987+0000","last_active":"2024-09-17T04:33:20.930207+0000","last_peered":"2024-09-17T04:33:20.930207+0000","last_clean":"2024-09-17T04:33:20.930207+0000","last_became_active":"2024-09-17T04:33:19.694651+0000","last_became_peered":"2024-09-17T04:33:19.694651+0000","last_unstale":"2024-09-17T04:33:20.930207+0000","last_undegraded":"2024-09-17T04:33:20.930207+0000","last_fullsized":"2024-09-17T04:33:20.930207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:57:44.135139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690540+0000","last_change":"2024-09-17T04:33:19.693592+0000","last_active":"2024-09-17T04:33:20.690540+0000","last_peered":"2024-09-17T04:33:20.690540+0000","last_clean":"2024-09-17T04:33:20.690540+0000","last_became_active":"2024-09-17T04:33:19.690470+0000","last_became_peered":"2024-09-17T04:33:19.690470+0000","last_unstale":"2024-09-17T04:33:20.690540+0000","last_undegraded":"2024-09-17T04:33:20.690540+0000","last_fullsized":"2024-09-17T04:33:20.690540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:16:38.450837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.929991+0000","last_change":"2024-09-17T04:33:19.696808+0000","last_active":"2024-09-17T04:33:20.929991+0000","last_peered":"2024-09-17T04:33:20.929991+0000","last_clean":"2024-09-17T04:33:20.929991+0000","last_became_active":"2024-09-17T04:33:19.696523+0000","last_became_peered":"2024-09-17T04:33:19.696523+0000","last_unstale":"2024-09-17T04:33:20.929991+0000","last_undegraded":"2024-09-17T04:33:20.929991+0000","last_fullsized":"2024-09-17T04:33:20.929991+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:21.722762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.689753+0000","last_change":"2024-09-17T04:33:19.698772+0000","last_active":"2024-09-17T04:33:20.689753+0000","last_peered":"2024-09-17T04:33:20.689753+0000","last_clean":"2024-09-17T04:33:20.689753+0000","last_became_active":"2024-09-17T04:33:19.698190+0000","last_became_peered":"2024-09-17T04:33:19.698190+0000","last_unstale":"2024-09-17T04:33:20.689753+0000","last_undegraded":"2024-09-17T04:33:20.689753+0000","last_fullsized":"2024-09-17T04:33:20.689753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:20.825186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035054+0000","last_change":"2024-09-17T04:33:19.702702+0000","last_active":"2024-09-17T04:33:22.035054+0000","last_peered":"2024-09-17T04:33:22.035054+0000","last_clean":"2024-09-17T04:33:22.035054+0000","last_became_active":"2024-09-17T04:33:19.701450+0000","last_became_peered":"2024-09-17T04:33:19.701450+0000","last_unstale":"2024-09-17T04:33:22.035054+0000","last_undegraded":"2024-09-17T04:33:22.035054+0000","last_fullsized":"2024-09-17T04:33:22.035054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:27.889015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691647+0000","last_change":"2024-09-17T04:33:19.702493+0000","last_active":"2024-09-17T04:33:20.691647+0000","last_peered":"2024-09-17T04:33:20.691647+0000","last_clean":"2024-09-17T04:33:20.691647+0000","last_became_active":"2024-09-17T04:33:19.702105+0000","last_became_peered":"2024-09-17T04:33:19.702105+0000","last_unstale":"2024-09-17T04:33:20.691647+0000","last_undegraded":"2024-09-17T04:33:20.691647+0000","last_fullsized":"2024-09-17T04:33:20.691647+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:21.242559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677475+0000","last_change":"2024-09-17T04:33:19.691454+0000","last_active":"2024-09-17T04:33:21.677475+0000","last_peered":"2024-09-17T04:33:21.677475+0000","last_clean":"2024-09-17T04:33:21.677475+0000","last_became_active":"2024-09-17T04:33:19.689654+0000","last_became_peered":"2024-09-17T04:33:19.689654+0000","last_unstale":"2024-09-17T04:33:21.677475+0000","last_undegraded":"2024-09-17T04:33:21.677475+0000","last_fullsized":"2024-09-17T04:33:21.677475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:49.699088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677244+0000","last_change":"2024-09-17T04:33:19.687918+0000","last_active":"2024-09-17T04:33:21.677244+0000","last_peered":"2024-09-17T04:33:21.677244+0000","last_clean":"2024-09-17T04:33:21.677244+0000","last_became_active":"2024-09-17T04:33:19.687575+0000","last_became_peered":"2024-09-17T04:33:19.687575+0000","last_unstale":"2024-09-17T04:33:21.677244+0000","last_undegraded":"2024-09-17T04:33:21.677244+0000","last_fullsized":"2024-09-17T04:33:21.677244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:39.340136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691463+0000","last_change":"2024-09-17T04:33:19.687282+0000","last_active":"2024-09-17T04:33:20.691463+0000","last_peered":"2024-09-17T04:33:20.691463+0000","last_clean":"2024-09-17T04:33:20.691463+0000","last_became_active":"2024-09-17T04:33:19.686308+0000","last_became_peered":"2024-09-17T04:33:19.686308+0000","last_unstale":"2024-09-17T04:33:20.691463+0000","last_undegraded":"2024-09-17T04:33:20.691463+0000","last_fullsized":"2024-09-17T04:33:20.691463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:05.880115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036019+0000","last_change":"2024-09-17T04:33:19.685671+0000","last_active":"2024-09-17T04:33:22.036019+0000","last_peered":"2024-09-17T04:33:22.036019+0000","last_clean":"2024-09-17T04:33:22.036019+0000","last_became_active":"2024-09-17T04:33:19.685339+0000","last_became_peered":"2024-09-17T04:33:19.685339+0000","last_unstale":"2024-09-17T04:33:22.036019+0000","last_undegraded":"2024-09-17T04:33:22.036019+0000","last_fullsized":"2024-09-17T04:33:22.036019+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:20.825186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036606+0000","last_change":"2024-09-17T04:33:19.682161+0000","last_active":"2024-09-17T04:33:22.036606+0000","last_peered":"2024-09-17T04:33:22.036606+0000","last_clean":"2024-09-17T04:33:22.036606+0000","last_became_active":"2024-09-17T04:33:19.679848+0000","last_became_peered":"2024-09-17T04:33:19.679848+0000","last_unstale":"2024-09-17T04:33:22.036606+0000","last_undegraded":"2024-09-17T04:33:22.036606+0000","last_fullsized":"2024-09-17T04:33:22.036606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:49.699088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674099+0000","last_change":"2024-09-17T04:33:19.681771+0000","last_active":"2024-09-17T04:33:21.674099+0000","last_peered":"2024-09-17T04:33:21.674099+0000","last_clean":"2024-09-17T04:33:21.674099+0000","last_became_active":"2024-09-17T04:33:19.681448+0000","last_became_peered":"2024-09-17T04:33:19.681448+0000","last_unstale":"2024-09-17T04:33:21.674099+0000","last_undegraded":"2024-09-17T04:33:21.674099+0000","last_fullsized":"2024-09-17T04:33:21.674099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:19.224546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.929925+0000","last_change":"2024-09-17T04:33:19.701067+0000","last_active":"2024-09-17T04:33:20.929925+0000","last_peered":"2024-09-17T04:33:20.929925+0000","last_clean":"2024-09-17T04:33:20.929925+0000","last_became_active":"2024-09-17T04:33:19.698312+0000","last_became_peered":"2024-09-17T04:33:19.698312+0000","last_unstale":"2024-09-17T04:33:20.929925+0000","last_undegraded":"2024-09-17T04:33:20.929925+0000","last_fullsized":"2024-09-17T04:33:20.929925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:49.699088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930842+0000","last_change":"2024-09-17T04:33:19.698864+0000","last_active":"2024-09-17T04:33:20.930842+0000","last_peered":"2024-09-17T04:33:20.930842+0000","last_clean":"2024-09-17T04:33:20.930842+0000","last_became_active":"2024-09-17T04:33:19.698394+0000","last_became_peered":"2024-09-17T04:33:19.698394+0000","last_unstale":"2024-09-17T04:33:20.930842+0000","last_undegraded":"2024-09-17T04:33:20.930842+0000","last_fullsized":"2024-09-17T04:33:20.930842+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:16:06.475955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.684437+0000","last_change":"2024-09-17T04:33:19.692126+0000","last_active":"2024-09-17T04:33:20.684437+0000","last_peered":"2024-09-17T04:33:20.684437+0000","last_clean":"2024-09-17T04:33:20.684437+0000","last_became_active":"2024-09-17T04:33:19.691759+0000","last_became_peered":"2024-09-17T04:33:19.691759+0000","last_unstale":"2024-09-17T04:33:20.684437+0000","last_undegraded":"2024-09-17T04:33:20.684437+0000","last_fullsized":"2024-09-17T04:33:20.684437+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:48.205935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690446+0000","last_change":"2024-09-17T04:33:19.693246+0000","last_active":"2024-09-17T04:33:20.690446+0000","last_peered":"2024-09-17T04:33:20.690446+0000","last_clean":"2024-09-17T04:33:20.690446+0000","last_became_active":"2024-09-17T04:33:19.692591+0000","last_became_peered":"2024-09-17T04:33:19.692591+0000","last_unstale":"2024-09-17T04:33:20.690446+0000","last_undegraded":"2024-09-17T04:33:20.690446+0000","last_fullsized":"2024-09-17T04:33:20.690446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:58.709184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677180+0000","last_change":"2024-09-17T04:33:19.684484+0000","last_active":"2024-09-17T04:33:21.677180+0000","last_peered":"2024-09-17T04:33:21.677180+0000","last_clean":"2024-09-17T04:33:21.677180+0000","last_became_active":"2024-09-17T04:33:19.683795+0000","last_became_peered":"2024-09-17T04:33:19.683795+0000","last_unstale":"2024-09-17T04:33:21.677180+0000","last_undegraded":"2024-09-17T04:33:21.677180+0000","last_fullsized":"2024-09-17T04:33:21.677180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:03.560173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036302+0000","last_change":"2024-09-17T04:33:19.691536+0000","last_active":"2024-09-17T04:33:22.036302+0000","last_peered":"2024-09-17T04:33:22.036302+0000","last_clean":"2024-09-17T04:33:22.036302+0000","last_became_active":"2024-09-17T04:33:19.691142+0000","last_became_peered":"2024-09-17T04:33:19.691142+0000","last_unstale":"2024-09-17T04:33:22.036302+0000","last_undegraded":"2024-09-17T04:33:22.036302+0000","last_fullsized":"2024-09-17T04:33:22.036302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:12.894886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930139+0000","last_change":"2024-09-17T04:33:19.700787+0000","last_active":"2024-09-17T04:33:20.930139+0000","last_peered":"2024-09-17T04:33:20.930139+0000","last_clean":"2024-09-17T04:33:20.930139+0000","last_became_active":"2024-09-17T04:33:19.700430+0000","last_became_peered":"2024-09-17T04:33:19.700430+0000","last_unstale":"2024-09-17T04:33:20.930139+0000","last_undegraded":"2024-09-17T04:33:20.930139+0000","last_fullsized":"2024-09-17T04:33:20.930139+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:14.652736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035743+0000","last_change":"2024-09-17T04:33:19.684011+0000","last_active":"2024-09-17T04:33:22.035743+0000","last_peered":"2024-09-17T04:33:22.035743+0000","last_clean":"2024-09-17T04:33:22.035743+0000","last_became_active":"2024-09-17T04:33:19.683695+0000","last_became_peered":"2024-09-17T04:33:19.683695+0000","last_unstale":"2024-09-17T04:33:22.035743+0000","last_undegraded":"2024-09-17T04:33:22.035743+0000","last_fullsized":"2024-09-17T04:33:22.035743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:11.769729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930009+0000","last_change":"2024-09-17T04:33:19.689591+0000","last_active":"2024-09-17T04:33:20.930009+0000","last_peered":"2024-09-17T04:33:20.930009+0000","last_clean":"2024-09-17T04:33:20.930009+0000","last_became_active":"2024-09-17T04:33:19.689263+0000","last_became_peered":"2024-09-17T04:33:19.689263+0000","last_unstale":"2024-09-17T04:33:20.930009+0000","last_undegraded":"2024-09-17T04:33:20.930009+0000","last_fullsized":"2024-09-17T04:33:20.930009+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:06:04.595656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035649+0000","last_change":"2024-09-17T04:33:19.689915+0000","last_active":"2024-09-17T04:33:22.035649+0000","last_peered":"2024-09-17T04:33:22.035649+0000","last_clean":"2024-09-17T04:33:22.035649+0000","last_became_active":"2024-09-17T04:33:19.689612+0000","last_became_peered":"2024-09-17T04:33:19.689612+0000","last_unstale":"2024-09-17T04:33:22.035649+0000","last_undegraded":"2024-09-17T04:33:22.035649+0000","last_fullsized":"2024-09-17T04:33:22.035649+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:39.340136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677482+0000","last_change":"2024-09-17T04:33:19.695306+0000","last_active":"2024-09-17T04:33:21.677482+0000","last_peered":"2024-09-17T04:33:21.677482+0000","last_clean":"2024-09-17T04:33:21.677482+0000","last_became_active":"2024-09-17T04:33:19.694980+0000","last_became_peered":"2024-09-17T04:33:19.694980+0000","last_unstale":"2024-09-17T04:33:21.677482+0000","last_undegraded":"2024-09-17T04:33:21.677482+0000","last_fullsized":"2024-09-17T04:33:21.677482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:11.769729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674250+0000","last_change":"2024-09-17T04:33:19.684015+0000","last_active":"2024-09-17T04:33:21.674250+0000","last_peered":"2024-09-17T04:33:21.674250+0000","last_clean":"2024-09-17T04:33:21.674250+0000","last_became_active":"2024-09-17T04:33:19.683668+0000","last_became_peered":"2024-09-17T04:33:19.683668+0000","last_unstale":"2024-09-17T04:33:21.674250+0000","last_undegraded":"2024-09-17T04:33:21.674250+0000","last_fullsized":"2024-09-17T04:33:21.674250+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:49.699088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035804+0000","last_change":"2024-09-17T04:33:19.692206+0000","last_active":"2024-09-17T04:33:22.035804+0000","last_peered":"2024-09-17T04:33:22.035804+0000","last_clean":"2024-09-17T04:33:22.035804+0000","last_became_active":"2024-09-17T04:33:19.691739+0000","last_became_peered":"2024-09-17T04:33:19.691739+0000","last_unstale":"2024-09-17T04:33:22.035804+0000","last_undegraded":"2024-09-17T04:33:22.035804+0000","last_fullsized":"2024-09-17T04:33:22.035804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:44.305489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036443+0000","last_change":"2024-09-17T04:33:19.688554+0000","last_active":"2024-09-17T04:33:22.036443+0000","last_peered":"2024-09-17T04:33:22.036443+0000","last_clean":"2024-09-17T04:33:22.036443+0000","last_became_active":"2024-09-17T04:33:19.688216+0000","last_became_peered":"2024-09-17T04:33:19.688216+0000","last_unstale":"2024-09-17T04:33:22.036443+0000","last_undegraded":"2024-09-17T04:33:22.036443+0000","last_fullsized":"2024-09-17T04:33:22.036443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:19.224546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930414+0000","last_change":"2024-09-17T04:33:19.690117+0000","last_active":"2024-09-17T04:33:20.930414+0000","last_peered":"2024-09-17T04:33:20.930414+0000","last_clean":"2024-09-17T04:33:20.930414+0000","last_became_active":"2024-09-17T04:33:19.689494+0000","last_became_peered":"2024-09-17T04:33:19.689494+0000","last_unstale":"2024-09-17T04:33:20.930414+0000","last_undegraded":"2024-09-17T04:33:20.930414+0000","last_fullsized":"2024-09-17T04:33:20.930414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:12.356559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.684135+0000","last_change":"2024-09-17T04:33:19.702680+0000","last_active":"2024-09-17T04:33:20.684135+0000","last_peered":"2024-09-17T04:33:20.684135+0000","last_clean":"2024-09-17T04:33:20.684135+0000","last_became_active":"2024-09-17T04:33:19.702240+0000","last_became_peered":"2024-09-17T04:33:19.702240+0000","last_unstale":"2024-09-17T04:33:20.684135+0000","last_undegraded":"2024-09-17T04:33:20.684135+0000","last_fullsized":"2024-09-17T04:33:20.684135+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:56:27.523263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677072+0000","last_change":"2024-09-17T04:33:19.698110+0000","last_active":"2024-09-17T04:33:21.677072+0000","last_peered":"2024-09-17T04:33:21.677072+0000","last_clean":"2024-09-17T04:33:21.677072+0000","last_became_active":"2024-09-17T04:33:19.697800+0000","last_became_peered":"2024-09-17T04:33:19.697800+0000","last_unstale":"2024-09-17T04:33:21.677072+0000","last_undegraded":"2024-09-17T04:33:21.677072+0000","last_fullsized":"2024-09-17T04:33:21.677072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:27.889015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.689882+0000","last_change":"2024-09-17T04:33:19.682605+0000","last_active":"2024-09-17T04:33:20.689882+0000","last_peered":"2024-09-17T04:33:20.689882+0000","last_clean":"2024-09-17T04:33:20.689882+0000","last_became_active":"2024-09-17T04:33:19.682212+0000","last_became_peered":"2024-09-17T04:33:19.682212+0000","last_unstale":"2024-09-17T04:33:20.689882+0000","last_undegraded":"2024-09-17T04:33:20.689882+0000","last_fullsized":"2024-09-17T04:33:20.689882+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:36:00.513397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036154+0000","last_change":"2024-09-17T04:33:18.690638+0000","last_active":"2024-09-17T04:33:22.036154+0000","last_peered":"2024-09-17T04:33:22.036154+0000","last_clean":"2024-09-17T04:33:22.036154+0000","last_became_active":"2024-09-17T04:33:18.690326+0000","last_became_peered":"2024-09-17T04:33:18.690326+0000","last_unstale":"2024-09-17T04:33:22.036154+0000","last_undegraded":"2024-09-17T04:33:22.036154+0000","last_fullsized":"2024-09-17T04:33:22.036154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:51.663061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036524+0000","last_change":"2024-09-17T04:33:19.686301+0000","last_active":"2024-09-17T04:33:22.036524+0000","last_peered":"2024-09-17T04:33:22.036524+0000","last_clean":"2024-09-17T04:33:22.036524+0000","last_became_active":"2024-09-17T04:33:19.684744+0000","last_became_peered":"2024-09-17T04:33:19.684744+0000","last_unstale":"2024-09-17T04:33:22.036524+0000","last_undegraded":"2024-09-17T04:33:22.036524+0000","last_fullsized":"2024-09-17T04:33:22.036524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:19:22.607029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.676940+0000","last_change":"2024-09-17T04:33:18.690509+0000","last_active":"2024-09-17T04:33:21.676940+0000","last_peered":"2024-09-17T04:33:21.676940+0000","last_clean":"2024-09-17T04:33:21.676940+0000","last_became_active":"2024-09-17T04:33:18.690113+0000","last_became_peered":"2024-09-17T04:33:18.690113+0000","last_unstale":"2024-09-17T04:33:21.676940+0000","last_undegraded":"2024-09-17T04:33:21.676940+0000","last_fullsized":"2024-09-17T04:33:21.676940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:14.180879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035866+0000","last_change":"2024-09-17T04:33:18.691233+0000","last_active":"2024-09-17T04:33:22.035866+0000","last_peered":"2024-09-17T04:33:22.035866+0000","last_clean":"2024-09-17T04:33:22.035866+0000","last_became_active":"2024-09-17T04:33:18.690790+0000","last_became_peered":"2024-09-17T04:33:18.690790+0000","last_unstale":"2024-09-17T04:33:22.035866+0000","last_undegraded":"2024-09-17T04:33:22.035866+0000","last_fullsized":"2024-09-17T04:33:22.035866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:14.180879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035122+0000","last_change":"2024-09-17T04:33:19.695559+0000","last_active":"2024-09-17T04:33:22.035122+0000","last_peered":"2024-09-17T04:33:22.035122+0000","last_clean":"2024-09-17T04:33:22.035122+0000","last_became_active":"2024-09-17T04:33:19.695236+0000","last_became_peered":"2024-09-17T04:33:19.695236+0000","last_unstale":"2024-09-17T04:33:22.035122+0000","last_undegraded":"2024-09-17T04:33:22.035122+0000","last_fullsized":"2024-09-17T04:33:22.035122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:19:59.773350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930110+0000","last_change":"2024-09-17T04:33:18.689567+0000","last_active":"2024-09-17T04:33:20.930110+0000","last_peered":"2024-09-17T04:33:20.930110+0000","last_clean":"2024-09-17T04:33:20.930110+0000","last_became_active":"2024-09-17T04:33:18.689202+0000","last_became_peered":"2024-09-17T04:33:18.689202+0000","last_unstale":"2024-09-17T04:33:20.930110+0000","last_undegraded":"2024-09-17T04:33:20.930110+0000","last_fullsized":"2024-09-17T04:33:20.930110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:23.445663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674031+0000","last_change":"2024-09-17T04:33:19.687275+0000","last_active":"2024-09-17T04:33:21.674031+0000","last_peered":"2024-09-17T04:33:21.674031+0000","last_clean":"2024-09-17T04:33:21.674031+0000","last_became_active":"2024-09-17T04:33:19.684741+0000","last_became_peered":"2024-09-17T04:33:19.684741+0000","last_unstale":"2024-09-17T04:33:21.674031+0000","last_undegraded":"2024-09-17T04:33:21.674031+0000","last_fullsized":"2024-09-17T04:33:21.674031+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:54:01.898864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677365+0000","last_change":"2024-09-17T04:33:18.694470+0000","last_active":"2024-09-17T04:33:21.677365+0000","last_peered":"2024-09-17T04:33:21.677365+0000","last_clean":"2024-09-17T04:33:21.677365+0000","last_became_active":"2024-09-17T04:33:18.694081+0000","last_became_peered":"2024-09-17T04:33:18.694081+0000","last_unstale":"2024-09-17T04:33:21.677365+0000","last_undegraded":"2024-09-17T04:33:21.677365+0000","last_fullsized":"2024-09-17T04:33:21.677365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:27.606246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930243+0000","last_change":"2024-09-17T04:33:19.695793+0000","last_active":"2024-09-17T04:33:20.930243+0000","last_peered":"2024-09-17T04:33:20.930243+0000","last_clean":"2024-09-17T04:33:20.930243+0000","last_became_active":"2024-09-17T04:33:19.695495+0000","last_became_peered":"2024-09-17T04:33:19.695495+0000","last_unstale":"2024-09-17T04:33:20.930243+0000","last_undegraded":"2024-09-17T04:33:20.930243+0000","last_fullsized":"2024-09-17T04:33:20.930243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:10:43.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930373+0000","last_change":"2024-09-17T04:33:18.694128+0000","last_active":"2024-09-17T04:33:20.930373+0000","last_peered":"2024-09-17T04:33:20.930373+0000","last_clean":"2024-09-17T04:33:20.930373+0000","last_became_active":"2024-09-17T04:33:18.693766+0000","last_became_peered":"2024-09-17T04:33:18.693766+0000","last_unstale":"2024-09-17T04:33:20.930373+0000","last_undegraded":"2024-09-17T04:33:20.930373+0000","last_fullsized":"2024-09-17T04:33:20.930373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:45:37.943381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036199+0000","last_change":"2024-09-17T04:33:19.691619+0000","last_active":"2024-09-17T04:33:22.036199+0000","last_peered":"2024-09-17T04:33:22.036199+0000","last_clean":"2024-09-17T04:33:22.036199+0000","last_became_active":"2024-09-17T04:33:19.690956+0000","last_became_peered":"2024-09-17T04:33:19.690956+0000","last_unstale":"2024-09-17T04:33:22.036199+0000","last_undegraded":"2024-09-17T04:33:22.036199+0000","last_fullsized":"2024-09-17T04:33:22.036199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:54:01.898864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674190+0000","last_change":"2024-09-17T04:33:18.691537+0000","last_active":"2024-09-17T04:33:21.674190+0000","last_peered":"2024-09-17T04:33:21.674190+0000","last_clean":"2024-09-17T04:33:21.674190+0000","last_became_active":"2024-09-17T04:33:18.691093+0000","last_became_peered":"2024-09-17T04:33:18.691093+0000","last_unstale":"2024-09-17T04:33:21.674190+0000","last_undegraded":"2024-09-17T04:33:21.674190+0000","last_fullsized":"2024-09-17T04:33:21.674190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:27.606246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691734+0000","last_change":"2024-09-17T04:33:19.691142+0000","last_active":"2024-09-17T04:33:20.691734+0000","last_peered":"2024-09-17T04:33:20.691734+0000","last_clean":"2024-09-17T04:33:20.691734+0000","last_became_active":"2024-09-17T04:33:19.690836+0000","last_became_peered":"2024-09-17T04:33:19.690836+0000","last_unstale":"2024-09-17T04:33:20.691734+0000","last_undegraded":"2024-09-17T04:33:20.691734+0000","last_fullsized":"2024-09-17T04:33:20.691734+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:36:40.026116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677278+0000","last_change":"2024-09-17T04:33:18.691686+0000","last_active":"2024-09-17T04:33:21.677278+0000","last_peered":"2024-09-17T04:33:21.677278+0000","last_clean":"2024-09-17T04:33:21.677278+0000","last_became_active":"2024-09-17T04:33:18.690643+0000","last_became_peered":"2024-09-17T04:33:18.690643+0000","last_unstale":"2024-09-17T04:33:21.677278+0000","last_undegraded":"2024-09-17T04:33:21.677278+0000","last_fullsized":"2024-09-17T04:33:21.677278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:53:28.012165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690245+0000","last_change":"2024-09-17T04:33:19.695036+0000","last_active":"2024-09-17T04:33:20.690245+0000","last_peered":"2024-09-17T04:33:20.690245+0000","last_clean":"2024-09-17T04:33:20.690245+0000","last_became_active":"2024-09-17T04:33:19.694670+0000","last_became_peered":"2024-09-17T04:33:19.694670+0000","last_unstale":"2024-09-17T04:33:20.690245+0000","last_undegraded":"2024-09-17T04:33:20.690245+0000","last_fullsized":"2024-09-17T04:33:20.690245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:44.199782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036569+0000","last_change":"2024-09-17T04:33:18.678408+0000","last_active":"2024-09-17T04:33:22.036569+0000","last_peered":"2024-09-17T04:33:22.036569+0000","last_clean":"2024-09-17T04:33:22.036569+0000","last_became_active":"2024-09-17T04:33:18.678054+0000","last_became_peered":"2024-09-17T04:33:18.678054+0000","last_unstale":"2024-09-17T04:33:22.036569+0000","last_undegraded":"2024-09-17T04:33:22.036569+0000","last_fullsized":"2024-09-17T04:33:22.036569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:51.663061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036608+0000","last_change":"2024-09-17T04:33:19.691672+0000","last_active":"2024-09-17T04:33:22.036608+0000","last_peered":"2024-09-17T04:33:22.036608+0000","last_clean":"2024-09-17T04:33:22.036608+0000","last_became_active":"2024-09-17T04:33:19.691319+0000","last_became_peered":"2024-09-17T04:33:19.691319+0000","last_unstale":"2024-09-17T04:33:22.036608+0000","last_undegraded":"2024-09-17T04:33:22.036608+0000","last_fullsized":"2024-09-17T04:33:22.036608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:44.305489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930312+0000","last_change":"2024-09-17T04:33:18.691730+0000","last_active":"2024-09-17T04:33:20.930312+0000","last_peered":"2024-09-17T04:33:20.930312+0000","last_clean":"2024-09-17T04:33:20.930312+0000","last_became_active":"2024-09-17T04:33:18.691046+0000","last_became_peered":"2024-09-17T04:33:18.691046+0000","last_unstale":"2024-09-17T04:33:20.930312+0000","last_undegraded":"2024-09-17T04:33:20.930312+0000","last_fullsized":"2024-09-17T04:33:20.930312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:53:28.012165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690746+0000","last_change":"2024-09-17T04:33:19.699326+0000","last_active":"2024-09-17T04:33:20.690746+0000","last_peered":"2024-09-17T04:33:20.690746+0000","last_clean":"2024-09-17T04:33:20.690746+0000","last_became_active":"2024-09-17T04:33:19.698549+0000","last_became_peered":"2024-09-17T04:33:19.698549+0000","last_unstale":"2024-09-17T04:33:20.690746+0000","last_undegraded":"2024-09-17T04:33:20.690746+0000","last_fullsized":"2024-09-17T04:33:20.690746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:28.772498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.676878+0000","last_change":"2024-09-17T04:33:18.691268+0000","last_active":"2024-09-17T04:33:21.676878+0000","last_peered":"2024-09-17T04:33:21.676878+0000","last_clean":"2024-09-17T04:33:21.676878+0000","last_became_active":"2024-09-17T04:33:18.690946+0000","last_became_peered":"2024-09-17T04:33:18.690946+0000","last_unstale":"2024-09-17T04:33:21.676878+0000","last_undegraded":"2024-09-17T04:33:21.676878+0000","last_fullsized":"2024-09-17T04:33:21.676878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:23.445663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677083+0000","last_change":"2024-09-17T04:33:19.695301+0000","last_active":"2024-09-17T04:33:21.677083+0000","last_peered":"2024-09-17T04:33:21.677083+0000","last_clean":"2024-09-17T04:33:21.677083+0000","last_became_active":"2024-09-17T04:33:19.694951+0000","last_became_peered":"2024-09-17T04:33:19.694951+0000","last_unstale":"2024-09-17T04:33:21.677083+0000","last_undegraded":"2024-09-17T04:33:21.677083+0000","last_fullsized":"2024-09-17T04:33:21.677083+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:22.696900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.683934+0000","last_change":"2024-09-17T04:33:18.684789+0000","last_active":"2024-09-17T04:33:20.683934+0000","last_peered":"2024-09-17T04:33:20.683934+0000","last_clean":"2024-09-17T04:33:20.683934+0000","last_became_active":"2024-09-17T04:33:18.684452+0000","last_became_peered":"2024-09-17T04:33:18.684452+0000","last_unstale":"2024-09-17T04:33:20.683934+0000","last_undegraded":"2024-09-17T04:33:20.683934+0000","last_fullsized":"2024-09-17T04:33:20.683934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:15:32.568929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690027+0000","last_change":"2024-09-17T04:33:19.699323+0000","last_active":"2024-09-17T04:33:20.690027+0000","last_peered":"2024-09-17T04:33:20.690027+0000","last_clean":"2024-09-17T04:33:20.690027+0000","last_became_active":"2024-09-17T04:33:19.698603+0000","last_became_peered":"2024-09-17T04:33:19.698603+0000","last_unstale":"2024-09-17T04:33:20.690027+0000","last_undegraded":"2024-09-17T04:33:20.690027+0000","last_fullsized":"2024-09-17T04:33:20.690027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:13.459991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691033+0000","last_change":"2024-09-17T04:33:18.681934+0000","last_active":"2024-09-17T04:33:20.691033+0000","last_peered":"2024-09-17T04:33:20.691033+0000","last_clean":"2024-09-17T04:33:20.691033+0000","last_became_active":"2024-09-17T04:33:18.681572+0000","last_became_peered":"2024-09-17T04:33:18.681572+0000","last_unstale":"2024-09-17T04:33:20.691033+0000","last_undegraded":"2024-09-17T04:33:20.691033+0000","last_fullsized":"2024-09-17T04:33:20.691033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:59.885891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691190+0000","last_change":"2024-09-17T04:33:19.699024+0000","last_active":"2024-09-17T04:33:20.691190+0000","last_peered":"2024-09-17T04:33:20.691190+0000","last_clean":"2024-09-17T04:33:20.691190+0000","last_became_active":"2024-09-17T04:33:19.698680+0000","last_became_peered":"2024-09-17T04:33:19.698680+0000","last_unstale":"2024-09-17T04:33:20.691190+0000","last_undegraded":"2024-09-17T04:33:20.691190+0000","last_fullsized":"2024-09-17T04:33:20.691190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:29:25.361534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.684435+0000","last_change":"2024-09-17T04:33:18.686803+0000","last_active":"2024-09-17T04:33:20.684435+0000","last_peered":"2024-09-17T04:33:20.684435+0000","last_clean":"2024-09-17T04:33:20.684435+0000","last_became_active":"2024-09-17T04:33:18.686293+0000","last_became_peered":"2024-09-17T04:33:18.686293+0000","last_unstale":"2024-09-17T04:33:20.684435+0000","last_undegraded":"2024-09-17T04:33:20.684435+0000","last_fullsized":"2024-09-17T04:33:20.684435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:07:39.351672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691823+0000","last_change":"2024-09-17T04:33:19.700000+0000","last_active":"2024-09-17T04:33:20.691823+0000","last_peered":"2024-09-17T04:33:20.691823+0000","last_clean":"2024-09-17T04:33:20.691823+0000","last_became_active":"2024-09-17T04:33:19.699695+0000","last_became_peered":"2024-09-17T04:33:19.699695+0000","last_unstale":"2024-09-17T04:33:20.691823+0000","last_undegraded":"2024-09-17T04:33:20.691823+0000","last_fullsized":"2024-09-17T04:33:20.691823+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:00.711952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674416+0000","last_change":"2024-09-17T04:33:18.684112+0000","last_active":"2024-09-17T04:33:21.674416+0000","last_peered":"2024-09-17T04:33:21.674416+0000","last_clean":"2024-09-17T04:33:21.674416+0000","last_became_active":"2024-09-17T04:33:18.683623+0000","last_became_peered":"2024-09-17T04:33:18.683623+0000","last_unstale":"2024-09-17T04:33:21.674416+0000","last_undegraded":"2024-09-17T04:33:21.674416+0000","last_fullsized":"2024-09-17T04:33:21.674416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:53.979348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930602+0000","last_change":"2024-09-17T04:33:19.688648+0000","last_active":"2024-09-17T04:33:20.930602+0000","last_peered":"2024-09-17T04:33:20.930602+0000","last_clean":"2024-09-17T04:33:20.930602+0000","last_became_active":"2024-09-17T04:33:19.688294+0000","last_became_peered":"2024-09-17T04:33:19.688294+0000","last_unstale":"2024-09-17T04:33:20.930602+0000","last_undegraded":"2024-09-17T04:33:20.930602+0000","last_fullsized":"2024-09-17T04:33:20.930602+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:26:38.002625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.684314+0000","last_change":"2024-09-17T04:33:18.690158+0000","last_active":"2024-09-17T04:33:20.684314+0000","last_peered":"2024-09-17T04:33:20.684314+0000","last_clean":"2024-09-17T04:33:20.684314+0000","last_became_active":"2024-09-17T04:33:18.689802+0000","last_became_peered":"2024-09-17T04:33:18.689802+0000","last_unstale":"2024-09-17T04:33:20.684314+0000","last_undegraded":"2024-09-17T04:33:20.684314+0000","last_fullsized":"2024-09-17T04:33:20.684314+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:21:44.098443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690295+0000","last_change":"2024-09-17T04:33:19.702563+0000","last_active":"2024-09-17T04:33:20.690295+0000","last_peered":"2024-09-17T04:33:20.690295+0000","last_clean":"2024-09-17T04:33:20.690295+0000","last_became_active":"2024-09-17T04:33:19.702241+0000","last_became_peered":"2024-09-17T04:33:19.702241+0000","last_unstale":"2024-09-17T04:33:20.690295+0000","last_undegraded":"2024-09-17T04:33:20.690295+0000","last_fullsized":"2024-09-17T04:33:20.690295+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:56:26.043321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690019+0000","last_change":"2024-09-17T04:33:09.978742+0000","last_active":"2024-09-17T04:33:20.690019+0000","last_peered":"2024-09-17T04:33:20.690019+0000","last_clean":"2024-09-17T04:33:20.690019+0000","last_became_active":"2024-09-17T04:33:09.977496+0000","last_became_peered":"2024-09-17T04:33:09.977496+0000","last_unstale":"2024-09-17T04:33:20.690019+0000","last_undegraded":"2024-09-17T04:33:20.690019+0000","last_fullsized":"2024-09-17T04:33:20.690019+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:08.930340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:08.930340+0000","last_clean_scrub_stamp":"2024-09-17T04:33:08.930340+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:21:26.566102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930309+0000","last_change":"2024-09-17T04:33:18.691736+0000","last_active":"2024-09-17T04:33:20.930309+0000","last_peered":"2024-09-17T04:33:20.930309+0000","last_clean":"2024-09-17T04:33:20.930309+0000","last_became_active":"2024-09-17T04:33:18.691410+0000","last_became_peered":"2024-09-17T04:33:18.691410+0000","last_unstale":"2024-09-17T04:33:20.930309+0000","last_undegraded":"2024-09-17T04:33:20.930309+0000","last_fullsized":"2024-09-17T04:33:20.930309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:14.180879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035352+0000","last_change":"2024-09-17T04:33:19.701468+0000","last_active":"2024-09-17T04:33:22.035352+0000","last_peered":"2024-09-17T04:33:22.035352+0000","last_clean":"2024-09-17T04:33:22.035352+0000","last_became_active":"2024-09-17T04:33:19.700764+0000","last_became_peered":"2024-09-17T04:33:19.700764+0000","last_unstale":"2024-09-17T04:33:22.035352+0000","last_undegraded":"2024-09-17T04:33:22.035352+0000","last_fullsized":"2024-09-17T04:33:22.035352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:16.814580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.689948+0000","last_change":"2024-09-17T04:33:18.687948+0000","last_active":"2024-09-17T04:33:20.689948+0000","last_peered":"2024-09-17T04:33:20.689948+0000","last_clean":"2024-09-17T04:33:20.689948+0000","last_became_active":"2024-09-17T04:33:18.687432+0000","last_became_peered":"2024-09-17T04:33:18.687432+0000","last_unstale":"2024-09-17T04:33:20.689948+0000","last_undegraded":"2024-09-17T04:33:20.689948+0000","last_fullsized":"2024-09-17T04:33:20.689948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:45:37.943381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035502+0000","last_change":"2024-09-17T04:33:19.687267+0000","last_active":"2024-09-17T04:33:22.035502+0000","last_peered":"2024-09-17T04:33:22.035502+0000","last_clean":"2024-09-17T04:33:22.035502+0000","last_became_active":"2024-09-17T04:33:19.684583+0000","last_became_peered":"2024-09-17T04:33:19.684583+0000","last_unstale":"2024-09-17T04:33:22.035502+0000","last_undegraded":"2024-09-17T04:33:22.035502+0000","last_fullsized":"2024-09-17T04:33:22.035502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:44.326971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.684040+0000","last_change":"2024-09-17T04:33:18.683812+0000","last_active":"2024-09-17T04:33:20.684040+0000","last_peered":"2024-09-17T04:33:20.684040+0000","last_clean":"2024-09-17T04:33:20.684040+0000","last_became_active":"2024-09-17T04:33:18.683253+0000","last_became_peered":"2024-09-17T04:33:18.683253+0000","last_unstale":"2024-09-17T04:33:20.684040+0000","last_undegraded":"2024-09-17T04:33:20.684040+0000","last_fullsized":"2024-09-17T04:33:20.684040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:00:10.198377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674050+0000","last_change":"2024-09-17T04:33:19.688660+0000","last_active":"2024-09-17T04:33:21.674050+0000","last_peered":"2024-09-17T04:33:21.674050+0000","last_clean":"2024-09-17T04:33:21.674050+0000","last_became_active":"2024-09-17T04:33:19.688336+0000","last_became_peered":"2024-09-17T04:33:19.688336+0000","last_unstale":"2024-09-17T04:33:21.674050+0000","last_undegraded":"2024-09-17T04:33:21.674050+0000","last_fullsized":"2024-09-17T04:33:21.674050+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:12.894886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930502+0000","last_change":"2024-09-17T04:33:18.688213+0000","last_active":"2024-09-17T04:33:20.930502+0000","last_peered":"2024-09-17T04:33:20.930502+0000","last_clean":"2024-09-17T04:33:20.930502+0000","last_became_active":"2024-09-17T04:33:18.687839+0000","last_became_peered":"2024-09-17T04:33:18.687839+0000","last_unstale":"2024-09-17T04:33:20.930502+0000","last_undegraded":"2024-09-17T04:33:20.930502+0000","last_fullsized":"2024-09-17T04:33:20.930502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:27.606246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690522+0000","last_change":"2024-09-17T04:33:19.695246+0000","last_active":"2024-09-17T04:33:20.690522+0000","last_peered":"2024-09-17T04:33:20.690522+0000","last_clean":"2024-09-17T04:33:20.690522+0000","last_became_active":"2024-09-17T04:33:19.694929+0000","last_became_peered":"2024-09-17T04:33:19.694929+0000","last_unstale":"2024-09-17T04:33:20.690522+0000","last_undegraded":"2024-09-17T04:33:20.690522+0000","last_fullsized":"2024-09-17T04:33:20.690522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:38:26.907899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674093+0000","last_change":"2024-09-17T04:33:18.682334+0000","last_active":"2024-09-17T04:33:21.674093+0000","last_peered":"2024-09-17T04:33:21.674093+0000","last_clean":"2024-09-17T04:33:21.674093+0000","last_became_active":"2024-09-17T04:33:18.681988+0000","last_became_peered":"2024-09-17T04:33:18.681988+0000","last_unstale":"2024-09-17T04:33:21.674093+0000","last_undegraded":"2024-09-17T04:33:21.674093+0000","last_fullsized":"2024-09-17T04:33:21.674093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:40.963166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930502+0000","last_change":"2024-09-17T04:33:19.694190+0000","last_active":"2024-09-17T04:33:20.930502+0000","last_peered":"2024-09-17T04:33:20.930502+0000","last_clean":"2024-09-17T04:33:20.930502+0000","last_became_active":"2024-09-17T04:33:19.691073+0000","last_became_peered":"2024-09-17T04:33:19.691073+0000","last_unstale":"2024-09-17T04:33:20.930502+0000","last_undegraded":"2024-09-17T04:33:20.930502+0000","last_fullsized":"2024-09-17T04:33:20.930502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:04:44.157056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690492+0000","last_change":"2024-09-17T04:33:18.687984+0000","last_active":"2024-09-17T04:33:20.690492+0000","last_peered":"2024-09-17T04:33:20.690492+0000","last_clean":"2024-09-17T04:33:20.690492+0000","last_became_active":"2024-09-17T04:33:18.687519+0000","last_became_peered":"2024-09-17T04:33:18.687519+0000","last_unstale":"2024-09-17T04:33:20.690492+0000","last_undegraded":"2024-09-17T04:33:20.690492+0000","last_fullsized":"2024-09-17T04:33:20.690492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:45.809989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674114+0000","last_change":"2024-09-17T04:33:19.688598+0000","last_active":"2024-09-17T04:33:21.674114+0000","last_peered":"2024-09-17T04:33:21.674114+0000","last_clean":"2024-09-17T04:33:21.674114+0000","last_became_active":"2024-09-17T04:33:19.688240+0000","last_became_peered":"2024-09-17T04:33:19.688240+0000","last_unstale":"2024-09-17T04:33:21.674114+0000","last_undegraded":"2024-09-17T04:33:21.674114+0000","last_fullsized":"2024-09-17T04:33:21.674114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:39.340136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.676358+0000","last_change":"2024-09-17T04:33:18.693000+0000","last_active":"2024-09-17T04:33:21.676358+0000","last_peered":"2024-09-17T04:33:21.676358+0000","last_clean":"2024-09-17T04:33:21.676358+0000","last_became_active":"2024-09-17T04:33:18.692677+0000","last_became_peered":"2024-09-17T04:33:18.692677+0000","last_unstale":"2024-09-17T04:33:21.676358+0000","last_undegraded":"2024-09-17T04:33:21.676358+0000","last_fullsized":"2024-09-17T04:33:21.676358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:45:37.943381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035886+0000","last_change":"2024-09-17T04:33:19.695017+0000","last_active":"2024-09-17T04:33:22.035886+0000","last_peered":"2024-09-17T04:33:22.035886+0000","last_clean":"2024-09-17T04:33:22.035886+0000","last_became_active":"2024-09-17T04:33:19.694621+0000","last_became_peered":"2024-09-17T04:33:19.694621+0000","last_unstale":"2024-09-17T04:33:22.035886+0000","last_undegraded":"2024-09-17T04:33:22.035886+0000","last_fullsized":"2024-09-17T04:33:22.035886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:38.326621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677397+0000","last_change":"2024-09-17T04:33:18.685926+0000","last_active":"2024-09-17T04:33:21.677397+0000","last_peered":"2024-09-17T04:33:21.677397+0000","last_clean":"2024-09-17T04:33:21.677397+0000","last_became_active":"2024-09-17T04:33:18.685598+0000","last_became_peered":"2024-09-17T04:33:18.685598+0000","last_unstale":"2024-09-17T04:33:21.677397+0000","last_undegraded":"2024-09-17T04:33:21.677397+0000","last_fullsized":"2024-09-17T04:33:21.677397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:45:37.943381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035362+0000","last_change":"2024-09-17T04:33:19.681836+0000","last_active":"2024-09-17T04:33:22.035362+0000","last_peered":"2024-09-17T04:33:22.035362+0000","last_clean":"2024-09-17T04:33:22.035362+0000","last_became_active":"2024-09-17T04:33:19.681085+0000","last_became_peered":"2024-09-17T04:33:19.681085+0000","last_unstale":"2024-09-17T04:33:22.035362+0000","last_undegraded":"2024-09-17T04:33:22.035362+0000","last_fullsized":"2024-09-17T04:33:22.035362+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:58.841388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.689553+0000","last_change":"2024-09-17T04:33:18.688337+0000","last_active":"2024-09-17T04:33:20.689553+0000","last_peered":"2024-09-17T04:33:20.689553+0000","last_clean":"2024-09-17T04:33:20.689553+0000","last_became_active":"2024-09-17T04:33:18.687331+0000","last_became_peered":"2024-09-17T04:33:18.687331+0000","last_unstale":"2024-09-17T04:33:20.689553+0000","last_undegraded":"2024-09-17T04:33:20.689553+0000","last_fullsized":"2024-09-17T04:33:20.689553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:27.606246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691952+0000","last_change":"2024-09-17T04:33:19.690673+0000","last_active":"2024-09-17T04:33:20.691952+0000","last_peered":"2024-09-17T04:33:20.691952+0000","last_clean":"2024-09-17T04:33:20.691952+0000","last_became_active":"2024-09-17T04:33:19.690356+0000","last_became_peered":"2024-09-17T04:33:19.690356+0000","last_unstale":"2024-09-17T04:33:20.691952+0000","last_undegraded":"2024-09-17T04:33:20.691952+0000","last_fullsized":"2024-09-17T04:33:20.691952+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:42.127433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674278+0000","last_change":"2024-09-17T04:33:18.686258+0000","last_active":"2024-09-17T04:33:21.674278+0000","last_peered":"2024-09-17T04:33:21.674278+0000","last_clean":"2024-09-17T04:33:21.674278+0000","last_became_active":"2024-09-17T04:33:18.685943+0000","last_became_peered":"2024-09-17T04:33:18.685943+0000","last_unstale":"2024-09-17T04:33:21.674278+0000","last_undegraded":"2024-09-17T04:33:21.674278+0000","last_fullsized":"2024-09-17T04:33:21.674278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:23.445663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035293+0000","last_change":"2024-09-17T04:33:19.684510+0000","last_active":"2024-09-17T04:33:22.035293+0000","last_peered":"2024-09-17T04:33:22.035293+0000","last_clean":"2024-09-17T04:33:22.035293+0000","last_became_active":"2024-09-17T04:33:19.683614+0000","last_became_peered":"2024-09-17T04:33:19.683614+0000","last_unstale":"2024-09-17T04:33:22.035293+0000","last_undegraded":"2024-09-17T04:33:22.035293+0000","last_fullsized":"2024-09-17T04:33:22.035293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:22.696900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674344+0000","last_change":"2024-09-17T04:33:18.684287+0000","last_active":"2024-09-17T04:33:21.674344+0000","last_peered":"2024-09-17T04:33:21.674344+0000","last_clean":"2024-09-17T04:33:21.674344+0000","last_became_active":"2024-09-17T04:33:18.683822+0000","last_became_peered":"2024-09-17T04:33:18.683822+0000","last_unstale":"2024-09-17T04:33:21.674344+0000","last_undegraded":"2024-09-17T04:33:21.674344+0000","last_fullsized":"2024-09-17T04:33:21.674344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:53:28.012165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036101+0000","last_change":"2024-09-17T04:33:19.699298+0000","last_active":"2024-09-17T04:33:22.036101+0000","last_peered":"2024-09-17T04:33:22.036101+0000","last_clean":"2024-09-17T04:33:22.036101+0000","last_became_active":"2024-09-17T04:33:19.698925+0000","last_became_peered":"2024-09-17T04:33:19.698925+0000","last_unstale":"2024-09-17T04:33:22.036101+0000","last_undegraded":"2024-09-17T04:33:22.036101+0000","last_fullsized":"2024-09-17T04:33:22.036101+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:13.459991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690689+0000","last_change":"2024-09-17T04:33:18.685941+0000","last_active":"2024-09-17T04:33:20.690689+0000","last_peered":"2024-09-17T04:33:20.690689+0000","last_clean":"2024-09-17T04:33:20.690689+0000","last_became_active":"2024-09-17T04:33:18.685484+0000","last_became_peered":"2024-09-17T04:33:18.685484+0000","last_unstale":"2024-09-17T04:33:20.690689+0000","last_undegraded":"2024-09-17T04:33:20.690689+0000","last_fullsized":"2024-09-17T04:33:20.690689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:40.963166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677177+0000","last_change":"2024-09-17T04:33:19.685032+0000","last_active":"2024-09-17T04:33:21.677177+0000","last_peered":"2024-09-17T04:33:21.677177+0000","last_clean":"2024-09-17T04:33:21.677177+0000","last_became_active":"2024-09-17T04:33:19.683930+0000","last_became_peered":"2024-09-17T04:33:19.683930+0000","last_unstale":"2024-09-17T04:33:21.677177+0000","last_undegraded":"2024-09-17T04:33:21.677177+0000","last_fullsized":"2024-09-17T04:33:21.677177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:16.814580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930467+0000","last_change":"2024-09-17T04:33:19.691762+0000","last_active":"2024-09-17T04:33:20.930467+0000","last_peered":"2024-09-17T04:33:20.930467+0000","last_clean":"2024-09-17T04:33:20.930467+0000","last_became_active":"2024-09-17T04:33:19.690765+0000","last_became_peered":"2024-09-17T04:33:19.690765+0000","last_unstale":"2024-09-17T04:33:20.930467+0000","last_undegraded":"2024-09-17T04:33:20.930467+0000","last_fullsized":"2024-09-17T04:33:20.930467+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:59.693472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.689803+0000","last_change":"2024-09-17T04:33:18.694221+0000","last_active":"2024-09-17T04:33:20.689803+0000","last_peered":"2024-09-17T04:33:20.689803+0000","last_clean":"2024-09-17T04:33:20.689803+0000","last_became_active":"2024-09-17T04:33:18.693906+0000","last_became_peered":"2024-09-17T04:33:18.693906+0000","last_unstale":"2024-09-17T04:33:20.689803+0000","last_undegraded":"2024-09-17T04:33:20.689803+0000","last_fullsized":"2024-09-17T04:33:20.689803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:53.979348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930455+0000","last_change":"2024-09-17T04:33:19.692952+0000","last_active":"2024-09-17T04:33:20.930455+0000","last_peered":"2024-09-17T04:33:20.930455+0000","last_clean":"2024-09-17T04:33:20.930455+0000","last_became_active":"2024-09-17T04:33:19.692612+0000","last_became_peered":"2024-09-17T04:33:19.692612+0000","last_unstale":"2024-09-17T04:33:20.930455+0000","last_undegraded":"2024-09-17T04:33:20.930455+0000","last_fullsized":"2024-09-17T04:33:20.930455+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:12:48.590039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690377+0000","last_change":"2024-09-17T04:33:18.692471+0000","last_active":"2024-09-17T04:33:20.690377+0000","last_peered":"2024-09-17T04:33:20.690377+0000","last_clean":"2024-09-17T04:33:20.690377+0000","last_became_active":"2024-09-17T04:33:18.692129+0000","last_became_peered":"2024-09-17T04:33:18.692129+0000","last_unstale":"2024-09-17T04:33:20.690377+0000","last_undegraded":"2024-09-17T04:33:20.690377+0000","last_fullsized":"2024-09-17T04:33:20.690377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:14:47.283001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035813+0000","last_change":"2024-09-17T04:33:19.691269+0000","last_active":"2024-09-17T04:33:22.035813+0000","last_peered":"2024-09-17T04:33:22.035813+0000","last_clean":"2024-09-17T04:33:22.035813+0000","last_became_active":"2024-09-17T04:33:19.690548+0000","last_became_peered":"2024-09-17T04:33:19.690548+0000","last_unstale":"2024-09-17T04:33:22.035813+0000","last_undegraded":"2024-09-17T04:33:22.035813+0000","last_fullsized":"2024-09-17T04:33:22.035813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:03.560173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036319+0000","last_change":"2024-09-17T04:33:18.689670+0000","last_active":"2024-09-17T04:33:22.036319+0000","last_peered":"2024-09-17T04:33:22.036319+0000","last_clean":"2024-09-17T04:33:22.036319+0000","last_became_active":"2024-09-17T04:33:18.689277+0000","last_became_peered":"2024-09-17T04:33:18.689277+0000","last_unstale":"2024-09-17T04:33:22.036319+0000","last_undegraded":"2024-09-17T04:33:22.036319+0000","last_fullsized":"2024-09-17T04:33:22.036319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:30.774768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036234+0000","last_change":"2024-09-17T04:33:19.686603+0000","last_active":"2024-09-17T04:33:22.036234+0000","last_peered":"2024-09-17T04:33:22.036234+0000","last_clean":"2024-09-17T04:33:22.036234+0000","last_became_active":"2024-09-17T04:33:19.686252+0000","last_became_peered":"2024-09-17T04:33:19.686252+0000","last_unstale":"2024-09-17T04:33:22.036234+0000","last_undegraded":"2024-09-17T04:33:22.036234+0000","last_fullsized":"2024-09-17T04:33:22.036234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:44.199782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035982+0000","last_change":"2024-09-17T04:33:18.683562+0000","last_active":"2024-09-17T04:33:22.035982+0000","last_peered":"2024-09-17T04:33:22.035982+0000","last_clean":"2024-09-17T04:33:22.035982+0000","last_became_active":"2024-09-17T04:33:18.683155+0000","last_became_peered":"2024-09-17T04:33:18.683155+0000","last_unstale":"2024-09-17T04:33:22.035982+0000","last_undegraded":"2024-09-17T04:33:22.035982+0000","last_fullsized":"2024-09-17T04:33:22.035982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:59:56.848238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542152,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":823296,"data_stored":669858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":80520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":78998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542152,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":237568,"data_stored":79034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542153,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":80520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542152,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":78908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":79068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":669914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:33:28.720 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:33:28.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.874+0000 7fcb4bea9640 1 -- 172.21.15.37:0/3059319701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb2c004100 msgr2=0x7fcb2c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.874+0000 7fcb4bea9640 1 --2- 172.21.15.37:0/3059319701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb2c004100 0x7fcb2c004800 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009b20 tx=0x7fcb2c005830 comp rx=0 tx=0).stop 2024-09-17T04:33:28.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.874+0000 7fcb4a446640 1 -- 172.21.15.37:0/3059319701 reap_dead start 2024-09-17T04:33:28.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.874+0000 7fcb4bea9640 1 -- 172.21.15.37:0/3059319701 shutdown_connections 2024-09-17T04:33:28.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.874+0000 7fcb4bea9640 1 -- 172.21.15.37:0/3059319701 >> 172.21.15.37:0/3059319701 conn(0x7fcb4406ddb0 msgr2=0x7fcb4406e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:28.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.874+0000 7fcb4bea9640 1 -- 172.21.15.37:0/3059319701 shutdown_connections 2024-09-17T04:33:28.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.874+0000 7fcb4bea9640 1 -- 172.21.15.37:0/3059319701 wait complete. 2024-09-17T04:33:28.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.876+0000 7fcb4bea9640 1 Processor -- start 2024-09-17T04:33:28.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.876+0000 7fcb4bea9640 1 -- start start 2024-09-17T04:33:28.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.877+0000 7fcb4bea9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb2c004100 0x7fcb440ab6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:28.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.877+0000 7fcb4bea9640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcb44152500 0x7fcb440abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:28.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.877+0000 7fcb4bea9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb441f88b0 0x7fcb441f8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:28.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.877+0000 7fcb4bea9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcb440d35f0 con 0x7fcb2c004100 2024-09-17T04:33:28.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.877+0000 7fcb4bea9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcb440d0f60 con 0x7fcb441f88b0 2024-09-17T04:33:28.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.877+0000 7fcb4bea9640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fcb440d3770 con 0x7fcb44152500 2024-09-17T04:33:28.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.877+0000 7fcb49c45640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb2c004100 0x7fcb440ab6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:28.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.877+0000 7fcb49c45640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb2c004100 0x7fcb440ab6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45936/0 (socket says 172.21.15.37:45936) 2024-09-17T04:33:28.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.877+0000 7fcb49c45640 1 -- 172.21.15.37:0/1855623110 learned_addr learned my addr 172.21.15.37:0/1855623110 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:28.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.877+0000 7fcb49444640 1 --2- 172.21.15.37:0/1855623110 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcb44152500 0x7fcb440abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:28.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.877+0000 7fcb4a446640 1 --2- 172.21.15.37:0/1855623110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb441f88b0 0x7fcb441f8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:28.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.877+0000 7fcb49c45640 1 -- 172.21.15.37:0/1855623110 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcb44152500 msgr2=0x7fcb440abc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.877+0000 7fcb49c45640 1 --2- 172.21.15.37:0/1855623110 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcb44152500 0x7fcb440abc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.877+0000 7fcb49c45640 1 -- 172.21.15.37:0/1855623110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb441f88b0 msgr2=0x7fcb441f8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:28.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.877+0000 7fcb49c45640 1 --2- 172.21.15.37:0/1855623110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb441f88b0 0x7fcb441f8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:28.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.877+0000 7fcb49c45640 1 -- 172.21.15.37:0/1855623110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb440b6020 con 0x7fcb2c004100 2024-09-17T04:33:28.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.878+0000 7fcb49c45640 1 --2- 172.21.15.37:0/1855623110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb2c004100 0x7fcb440ab6e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcb34006090 tx=0x7fcb34006550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:28.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.878+0000 7fcb427fc640 1 -- 172.21.15.37:0/1855623110 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb34015070 con 0x7fcb2c004100 2024-09-17T04:33:28.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.878+0000 7fcb427fc640 1 -- 172.21.15.37:0/1855623110 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb3400fa90 con 0x7fcb2c004100 2024-09-17T04:33:28.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.878+0000 7fcb4bea9640 1 -- 172.21.15.37:0/1855623110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb440b6300 con 0x7fcb2c004100 2024-09-17T04:33:28.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.878+0000 7fcb427fc640 1 -- 172.21.15.37:0/1855623110 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb34011430 con 0x7fcb2c004100 2024-09-17T04:33:28.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.878+0000 7fcb4bea9640 1 -- 172.21.15.37:0/1855623110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb440ab1c0 con 0x7fcb2c004100 2024-09-17T04:33:28.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.879+0000 7fcb4bea9640 1 -- 172.21.15.37:0/1855623110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb440d3cd0 con 0x7fcb2c004100 2024-09-17T04:33:28.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.880+0000 7fcb427fc640 1 -- 172.21.15.37:0/1855623110 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcb340115d0 con 0x7fcb2c004100 2024-09-17T04:33:28.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.880+0000 7fcb427fc640 1 --2- 172.21.15.37:0/1855623110 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fcb30078b90 0x7fcb3007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:28.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.880+0000 7fcb427fc640 1 -- 172.21.15.37:0/1855623110 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fcb3409bc50 con 0x7fcb2c004100 2024-09-17T04:33:28.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.881+0000 7fcb49444640 1 --2- 172.21.15.37:0/1855623110 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fcb30078b90 0x7fcb3007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:28.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.881+0000 7fcb49444640 1 --2- 172.21.15.37:0/1855623110 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fcb30078b90 0x7fcb3007b0a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcb38035000 tx=0x7fcb380046c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:28.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:28.885+0000 7fcb427fc640 1 -- 172.21.15.37:0/1855623110 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fcb34064080 con 0x7fcb2c004100 2024-09-17T04:33:29.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.066+0000 7fcb4bea9640 1 -- 172.21.15.37:0/1855623110 --> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcb4406e630 con 0x7fcb30078b90 2024-09-17T04:33:29.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.071+0000 7fcb427fc640 1 -- 172.21.15.37:0/1855623110 <== mgr.4100 v2:172.21.15.92:6800/864337835 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251825 (secure 0 0 0) 0x7fcb4406e630 con 0x7fcb30078b90 2024-09-17T04:33:29.073 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:29.074 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-17T04:33:29.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.074+0000 7fcb4bea9640 1 -- 172.21.15.37:0/1855623110 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fcb30078b90 msgr2=0x7fcb3007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:29.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.074+0000 7fcb4bea9640 1 --2- 172.21.15.37:0/1855623110 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fcb30078b90 0x7fcb3007b0a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcb38035000 tx=0x7fcb380046c0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.074+0000 7fcb4bea9640 1 -- 172.21.15.37:0/1855623110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb2c004100 msgr2=0x7fcb440ab6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:29.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.074+0000 7fcb4bea9640 1 --2- 172.21.15.37:0/1855623110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb2c004100 0x7fcb440ab6e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcb34006090 tx=0x7fcb34006550 comp rx=0 tx=0).stop 2024-09-17T04:33:29.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.074+0000 7fcb4bea9640 1 -- 172.21.15.37:0/1855623110 shutdown_connections 2024-09-17T04:33:29.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.074+0000 7fcb4bea9640 1 --2- 172.21.15.37:0/1855623110 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fcb30078b90 0x7fcb3007b0a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.074+0000 7fcb4bea9640 1 --2- 172.21.15.37:0/1855623110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb441f88b0 0x7fcb441f8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.074+0000 7fcb4bea9640 1 --2- 172.21.15.37:0/1855623110 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcb44152500 0x7fcb440abc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.074+0000 7fcb4bea9640 1 --2- 172.21.15.37:0/1855623110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb2c004100 0x7fcb440ab6e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.074+0000 7fcb4bea9640 1 -- 172.21.15.37:0/1855623110 >> 172.21.15.37:0/1855623110 conn(0x7fcb4406ddb0 msgr2=0x7fcb440cf1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:29.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.074+0000 7fcb4bea9640 1 -- 172.21.15.37:0/1855623110 shutdown_connections 2024-09-17T04:33:29.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.074+0000 7fcb4bea9640 1 -- 172.21.15.37:0/1855623110 wait complete. 2024-09-17T04:33:29.085 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-17T04:33:28.447609+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219216,"kb_used_data":2984,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755504,"statfs":{"total":773094113280,"available":772869636096,"internally_reserved":0,"allocated":3055616,"data_stored":1816820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.777989"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691577+0000","last_change":"2024-09-17T04:33:19.693609+0000","last_active":"2024-09-17T04:33:20.691577+0000","last_peered":"2024-09-17T04:33:20.691577+0000","last_clean":"2024-09-17T04:33:20.691577+0000","last_became_active":"2024-09-17T04:33:19.693159+0000","last_became_peered":"2024-09-17T04:33:19.693159+0000","last_unstale":"2024-09-17T04:33:20.691577+0000","last_undegraded":"2024-09-17T04:33:20.691577+0000","last_fullsized":"2024-09-17T04:33:20.691577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:29.058321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674309+0000","last_change":"2024-09-17T04:33:19.686724+0000","last_active":"2024-09-17T04:33:21.674309+0000","last_peered":"2024-09-17T04:33:21.674309+0000","last_clean":"2024-09-17T04:33:21.674309+0000","last_became_active":"2024-09-17T04:33:19.685840+0000","last_became_peered":"2024-09-17T04:33:19.685840+0000","last_unstale":"2024-09-17T04:33:21.674309+0000","last_undegraded":"2024-09-17T04:33:21.674309+0000","last_fullsized":"2024-09-17T04:33:21.674309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:21.722762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035198+0000","last_change":"2024-09-17T04:33:19.688324+0000","last_active":"2024-09-17T04:33:22.035198+0000","last_peered":"2024-09-17T04:33:22.035198+0000","last_clean":"2024-09-17T04:33:22.035198+0000","last_became_active":"2024-09-17T04:33:19.687914+0000","last_became_peered":"2024-09-17T04:33:19.687914+0000","last_unstale":"2024-09-17T04:33:22.035198+0000","last_undegraded":"2024-09-17T04:33:22.035198+0000","last_fullsized":"2024-09-17T04:33:22.035198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:44.753208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930207+0000","last_change":"2024-09-17T04:33:19.694987+0000","last_active":"2024-09-17T04:33:20.930207+0000","last_peered":"2024-09-17T04:33:20.930207+0000","last_clean":"2024-09-17T04:33:20.930207+0000","last_became_active":"2024-09-17T04:33:19.694651+0000","last_became_peered":"2024-09-17T04:33:19.694651+0000","last_unstale":"2024-09-17T04:33:20.930207+0000","last_undegraded":"2024-09-17T04:33:20.930207+0000","last_fullsized":"2024-09-17T04:33:20.930207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:57:44.135139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690540+0000","last_change":"2024-09-17T04:33:19.693592+0000","last_active":"2024-09-17T04:33:20.690540+0000","last_peered":"2024-09-17T04:33:20.690540+0000","last_clean":"2024-09-17T04:33:20.690540+0000","last_became_active":"2024-09-17T04:33:19.690470+0000","last_became_peered":"2024-09-17T04:33:19.690470+0000","last_unstale":"2024-09-17T04:33:20.690540+0000","last_undegraded":"2024-09-17T04:33:20.690540+0000","last_fullsized":"2024-09-17T04:33:20.690540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:16:38.450837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.929991+0000","last_change":"2024-09-17T04:33:19.696808+0000","last_active":"2024-09-17T04:33:20.929991+0000","last_peered":"2024-09-17T04:33:20.929991+0000","last_clean":"2024-09-17T04:33:20.929991+0000","last_became_active":"2024-09-17T04:33:19.696523+0000","last_became_peered":"2024-09-17T04:33:19.696523+0000","last_unstale":"2024-09-17T04:33:20.929991+0000","last_undegraded":"2024-09-17T04:33:20.929991+0000","last_fullsized":"2024-09-17T04:33:20.929991+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:21.722762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.689753+0000","last_change":"2024-09-17T04:33:19.698772+0000","last_active":"2024-09-17T04:33:20.689753+0000","last_peered":"2024-09-17T04:33:20.689753+0000","last_clean":"2024-09-17T04:33:20.689753+0000","last_became_active":"2024-09-17T04:33:19.698190+0000","last_became_peered":"2024-09-17T04:33:19.698190+0000","last_unstale":"2024-09-17T04:33:20.689753+0000","last_undegraded":"2024-09-17T04:33:20.689753+0000","last_fullsized":"2024-09-17T04:33:20.689753+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:20.825186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035054+0000","last_change":"2024-09-17T04:33:19.702702+0000","last_active":"2024-09-17T04:33:22.035054+0000","last_peered":"2024-09-17T04:33:22.035054+0000","last_clean":"2024-09-17T04:33:22.035054+0000","last_became_active":"2024-09-17T04:33:19.701450+0000","last_became_peered":"2024-09-17T04:33:19.701450+0000","last_unstale":"2024-09-17T04:33:22.035054+0000","last_undegraded":"2024-09-17T04:33:22.035054+0000","last_fullsized":"2024-09-17T04:33:22.035054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:27.889015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691647+0000","last_change":"2024-09-17T04:33:19.702493+0000","last_active":"2024-09-17T04:33:20.691647+0000","last_peered":"2024-09-17T04:33:20.691647+0000","last_clean":"2024-09-17T04:33:20.691647+0000","last_became_active":"2024-09-17T04:33:19.702105+0000","last_became_peered":"2024-09-17T04:33:19.702105+0000","last_unstale":"2024-09-17T04:33:20.691647+0000","last_undegraded":"2024-09-17T04:33:20.691647+0000","last_fullsized":"2024-09-17T04:33:20.691647+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:21.242559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677475+0000","last_change":"2024-09-17T04:33:19.691454+0000","last_active":"2024-09-17T04:33:21.677475+0000","last_peered":"2024-09-17T04:33:21.677475+0000","last_clean":"2024-09-17T04:33:21.677475+0000","last_became_active":"2024-09-17T04:33:19.689654+0000","last_became_peered":"2024-09-17T04:33:19.689654+0000","last_unstale":"2024-09-17T04:33:21.677475+0000","last_undegraded":"2024-09-17T04:33:21.677475+0000","last_fullsized":"2024-09-17T04:33:21.677475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:49.699088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677244+0000","last_change":"2024-09-17T04:33:19.687918+0000","last_active":"2024-09-17T04:33:21.677244+0000","last_peered":"2024-09-17T04:33:21.677244+0000","last_clean":"2024-09-17T04:33:21.677244+0000","last_became_active":"2024-09-17T04:33:19.687575+0000","last_became_peered":"2024-09-17T04:33:19.687575+0000","last_unstale":"2024-09-17T04:33:21.677244+0000","last_undegraded":"2024-09-17T04:33:21.677244+0000","last_fullsized":"2024-09-17T04:33:21.677244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:39.340136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691463+0000","last_change":"2024-09-17T04:33:19.687282+0000","last_active":"2024-09-17T04:33:20.691463+0000","last_peered":"2024-09-17T04:33:20.691463+0000","last_clean":"2024-09-17T04:33:20.691463+0000","last_became_active":"2024-09-17T04:33:19.686308+0000","last_became_peered":"2024-09-17T04:33:19.686308+0000","last_unstale":"2024-09-17T04:33:20.691463+0000","last_undegraded":"2024-09-17T04:33:20.691463+0000","last_fullsized":"2024-09-17T04:33:20.691463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:05.880115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036019+0000","last_change":"2024-09-17T04:33:19.685671+0000","last_active":"2024-09-17T04:33:22.036019+0000","last_peered":"2024-09-17T04:33:22.036019+0000","last_clean":"2024-09-17T04:33:22.036019+0000","last_became_active":"2024-09-17T04:33:19.685339+0000","last_became_peered":"2024-09-17T04:33:19.685339+0000","last_unstale":"2024-09-17T04:33:22.036019+0000","last_undegraded":"2024-09-17T04:33:22.036019+0000","last_fullsized":"2024-09-17T04:33:22.036019+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:20.825186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036606+0000","last_change":"2024-09-17T04:33:19.682161+0000","last_active":"2024-09-17T04:33:22.036606+0000","last_peered":"2024-09-17T04:33:22.036606+0000","last_clean":"2024-09-17T04:33:22.036606+0000","last_became_active":"2024-09-17T04:33:19.679848+0000","last_became_peered":"2024-09-17T04:33:19.679848+0000","last_unstale":"2024-09-17T04:33:22.036606+0000","last_undegraded":"2024-09-17T04:33:22.036606+0000","last_fullsized":"2024-09-17T04:33:22.036606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:49.699088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674099+0000","last_change":"2024-09-17T04:33:19.681771+0000","last_active":"2024-09-17T04:33:21.674099+0000","last_peered":"2024-09-17T04:33:21.674099+0000","last_clean":"2024-09-17T04:33:21.674099+0000","last_became_active":"2024-09-17T04:33:19.681448+0000","last_became_peered":"2024-09-17T04:33:19.681448+0000","last_unstale":"2024-09-17T04:33:21.674099+0000","last_undegraded":"2024-09-17T04:33:21.674099+0000","last_fullsized":"2024-09-17T04:33:21.674099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:19.224546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.929925+0000","last_change":"2024-09-17T04:33:19.701067+0000","last_active":"2024-09-17T04:33:20.929925+0000","last_peered":"2024-09-17T04:33:20.929925+0000","last_clean":"2024-09-17T04:33:20.929925+0000","last_became_active":"2024-09-17T04:33:19.698312+0000","last_became_peered":"2024-09-17T04:33:19.698312+0000","last_unstale":"2024-09-17T04:33:20.929925+0000","last_undegraded":"2024-09-17T04:33:20.929925+0000","last_fullsized":"2024-09-17T04:33:20.929925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:49.699088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930842+0000","last_change":"2024-09-17T04:33:19.698864+0000","last_active":"2024-09-17T04:33:20.930842+0000","last_peered":"2024-09-17T04:33:20.930842+0000","last_clean":"2024-09-17T04:33:20.930842+0000","last_became_active":"2024-09-17T04:33:19.698394+0000","last_became_peered":"2024-09-17T04:33:19.698394+0000","last_unstale":"2024-09-17T04:33:20.930842+0000","last_undegraded":"2024-09-17T04:33:20.930842+0000","last_fullsized":"2024-09-17T04:33:20.930842+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:16:06.475955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.684437+0000","last_change":"2024-09-17T04:33:19.692126+0000","last_active":"2024-09-17T04:33:20.684437+0000","last_peered":"2024-09-17T04:33:20.684437+0000","last_clean":"2024-09-17T04:33:20.684437+0000","last_became_active":"2024-09-17T04:33:19.691759+0000","last_became_peered":"2024-09-17T04:33:19.691759+0000","last_unstale":"2024-09-17T04:33:20.684437+0000","last_undegraded":"2024-09-17T04:33:20.684437+0000","last_fullsized":"2024-09-17T04:33:20.684437+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:48.205935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690446+0000","last_change":"2024-09-17T04:33:19.693246+0000","last_active":"2024-09-17T04:33:20.690446+0000","last_peered":"2024-09-17T04:33:20.690446+0000","last_clean":"2024-09-17T04:33:20.690446+0000","last_became_active":"2024-09-17T04:33:19.692591+0000","last_became_peered":"2024-09-17T04:33:19.692591+0000","last_unstale":"2024-09-17T04:33:20.690446+0000","last_undegraded":"2024-09-17T04:33:20.690446+0000","last_fullsized":"2024-09-17T04:33:20.690446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:58.709184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677180+0000","last_change":"2024-09-17T04:33:19.684484+0000","last_active":"2024-09-17T04:33:21.677180+0000","last_peered":"2024-09-17T04:33:21.677180+0000","last_clean":"2024-09-17T04:33:21.677180+0000","last_became_active":"2024-09-17T04:33:19.683795+0000","last_became_peered":"2024-09-17T04:33:19.683795+0000","last_unstale":"2024-09-17T04:33:21.677180+0000","last_undegraded":"2024-09-17T04:33:21.677180+0000","last_fullsized":"2024-09-17T04:33:21.677180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:03.560173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036302+0000","last_change":"2024-09-17T04:33:19.691536+0000","last_active":"2024-09-17T04:33:22.036302+0000","last_peered":"2024-09-17T04:33:22.036302+0000","last_clean":"2024-09-17T04:33:22.036302+0000","last_became_active":"2024-09-17T04:33:19.691142+0000","last_became_peered":"2024-09-17T04:33:19.691142+0000","last_unstale":"2024-09-17T04:33:22.036302+0000","last_undegraded":"2024-09-17T04:33:22.036302+0000","last_fullsized":"2024-09-17T04:33:22.036302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:12.894886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930139+0000","last_change":"2024-09-17T04:33:19.700787+0000","last_active":"2024-09-17T04:33:20.930139+0000","last_peered":"2024-09-17T04:33:20.930139+0000","last_clean":"2024-09-17T04:33:20.930139+0000","last_became_active":"2024-09-17T04:33:19.700430+0000","last_became_peered":"2024-09-17T04:33:19.700430+0000","last_unstale":"2024-09-17T04:33:20.930139+0000","last_undegraded":"2024-09-17T04:33:20.930139+0000","last_fullsized":"2024-09-17T04:33:20.930139+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:14.652736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035743+0000","last_change":"2024-09-17T04:33:19.684011+0000","last_active":"2024-09-17T04:33:22.035743+0000","last_peered":"2024-09-17T04:33:22.035743+0000","last_clean":"2024-09-17T04:33:22.035743+0000","last_became_active":"2024-09-17T04:33:19.683695+0000","last_became_peered":"2024-09-17T04:33:19.683695+0000","last_unstale":"2024-09-17T04:33:22.035743+0000","last_undegraded":"2024-09-17T04:33:22.035743+0000","last_fullsized":"2024-09-17T04:33:22.035743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:11.769729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930009+0000","last_change":"2024-09-17T04:33:19.689591+0000","last_active":"2024-09-17T04:33:20.930009+0000","last_peered":"2024-09-17T04:33:20.930009+0000","last_clean":"2024-09-17T04:33:20.930009+0000","last_became_active":"2024-09-17T04:33:19.689263+0000","last_became_peered":"2024-09-17T04:33:19.689263+0000","last_unstale":"2024-09-17T04:33:20.930009+0000","last_undegraded":"2024-09-17T04:33:20.930009+0000","last_fullsized":"2024-09-17T04:33:20.930009+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:06:04.595656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035649+0000","last_change":"2024-09-17T04:33:19.689915+0000","last_active":"2024-09-17T04:33:22.035649+0000","last_peered":"2024-09-17T04:33:22.035649+0000","last_clean":"2024-09-17T04:33:22.035649+0000","last_became_active":"2024-09-17T04:33:19.689612+0000","last_became_peered":"2024-09-17T04:33:19.689612+0000","last_unstale":"2024-09-17T04:33:22.035649+0000","last_undegraded":"2024-09-17T04:33:22.035649+0000","last_fullsized":"2024-09-17T04:33:22.035649+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:39.340136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677482+0000","last_change":"2024-09-17T04:33:19.695306+0000","last_active":"2024-09-17T04:33:21.677482+0000","last_peered":"2024-09-17T04:33:21.677482+0000","last_clean":"2024-09-17T04:33:21.677482+0000","last_became_active":"2024-09-17T04:33:19.694980+0000","last_became_peered":"2024-09-17T04:33:19.694980+0000","last_unstale":"2024-09-17T04:33:21.677482+0000","last_undegraded":"2024-09-17T04:33:21.677482+0000","last_fullsized":"2024-09-17T04:33:21.677482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:11.769729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674250+0000","last_change":"2024-09-17T04:33:19.684015+0000","last_active":"2024-09-17T04:33:21.674250+0000","last_peered":"2024-09-17T04:33:21.674250+0000","last_clean":"2024-09-17T04:33:21.674250+0000","last_became_active":"2024-09-17T04:33:19.683668+0000","last_became_peered":"2024-09-17T04:33:19.683668+0000","last_unstale":"2024-09-17T04:33:21.674250+0000","last_undegraded":"2024-09-17T04:33:21.674250+0000","last_fullsized":"2024-09-17T04:33:21.674250+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:49.699088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035804+0000","last_change":"2024-09-17T04:33:19.692206+0000","last_active":"2024-09-17T04:33:22.035804+0000","last_peered":"2024-09-17T04:33:22.035804+0000","last_clean":"2024-09-17T04:33:22.035804+0000","last_became_active":"2024-09-17T04:33:19.691739+0000","last_became_peered":"2024-09-17T04:33:19.691739+0000","last_unstale":"2024-09-17T04:33:22.035804+0000","last_undegraded":"2024-09-17T04:33:22.035804+0000","last_fullsized":"2024-09-17T04:33:22.035804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:44.305489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036443+0000","last_change":"2024-09-17T04:33:19.688554+0000","last_active":"2024-09-17T04:33:22.036443+0000","last_peered":"2024-09-17T04:33:22.036443+0000","last_clean":"2024-09-17T04:33:22.036443+0000","last_became_active":"2024-09-17T04:33:19.688216+0000","last_became_peered":"2024-09-17T04:33:19.688216+0000","last_unstale":"2024-09-17T04:33:22.036443+0000","last_undegraded":"2024-09-17T04:33:22.036443+0000","last_fullsized":"2024-09-17T04:33:22.036443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:19.224546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930414+0000","last_change":"2024-09-17T04:33:19.690117+0000","last_active":"2024-09-17T04:33:20.930414+0000","last_peered":"2024-09-17T04:33:20.930414+0000","last_clean":"2024-09-17T04:33:20.930414+0000","last_became_active":"2024-09-17T04:33:19.689494+0000","last_became_peered":"2024-09-17T04:33:19.689494+0000","last_unstale":"2024-09-17T04:33:20.930414+0000","last_undegraded":"2024-09-17T04:33:20.930414+0000","last_fullsized":"2024-09-17T04:33:20.930414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:12.356559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.684135+0000","last_change":"2024-09-17T04:33:19.702680+0000","last_active":"2024-09-17T04:33:20.684135+0000","last_peered":"2024-09-17T04:33:20.684135+0000","last_clean":"2024-09-17T04:33:20.684135+0000","last_became_active":"2024-09-17T04:33:19.702240+0000","last_became_peered":"2024-09-17T04:33:19.702240+0000","last_unstale":"2024-09-17T04:33:20.684135+0000","last_undegraded":"2024-09-17T04:33:20.684135+0000","last_fullsized":"2024-09-17T04:33:20.684135+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:56:27.523263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677072+0000","last_change":"2024-09-17T04:33:19.698110+0000","last_active":"2024-09-17T04:33:21.677072+0000","last_peered":"2024-09-17T04:33:21.677072+0000","last_clean":"2024-09-17T04:33:21.677072+0000","last_became_active":"2024-09-17T04:33:19.697800+0000","last_became_peered":"2024-09-17T04:33:19.697800+0000","last_unstale":"2024-09-17T04:33:21.677072+0000","last_undegraded":"2024-09-17T04:33:21.677072+0000","last_fullsized":"2024-09-17T04:33:21.677072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:27.889015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.689882+0000","last_change":"2024-09-17T04:33:19.682605+0000","last_active":"2024-09-17T04:33:20.689882+0000","last_peered":"2024-09-17T04:33:20.689882+0000","last_clean":"2024-09-17T04:33:20.689882+0000","last_became_active":"2024-09-17T04:33:19.682212+0000","last_became_peered":"2024-09-17T04:33:19.682212+0000","last_unstale":"2024-09-17T04:33:20.689882+0000","last_undegraded":"2024-09-17T04:33:20.689882+0000","last_fullsized":"2024-09-17T04:33:20.689882+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:36:00.513397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036154+0000","last_change":"2024-09-17T04:33:18.690638+0000","last_active":"2024-09-17T04:33:22.036154+0000","last_peered":"2024-09-17T04:33:22.036154+0000","last_clean":"2024-09-17T04:33:22.036154+0000","last_became_active":"2024-09-17T04:33:18.690326+0000","last_became_peered":"2024-09-17T04:33:18.690326+0000","last_unstale":"2024-09-17T04:33:22.036154+0000","last_undegraded":"2024-09-17T04:33:22.036154+0000","last_fullsized":"2024-09-17T04:33:22.036154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:51.663061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036524+0000","last_change":"2024-09-17T04:33:19.686301+0000","last_active":"2024-09-17T04:33:22.036524+0000","last_peered":"2024-09-17T04:33:22.036524+0000","last_clean":"2024-09-17T04:33:22.036524+0000","last_became_active":"2024-09-17T04:33:19.684744+0000","last_became_peered":"2024-09-17T04:33:19.684744+0000","last_unstale":"2024-09-17T04:33:22.036524+0000","last_undegraded":"2024-09-17T04:33:22.036524+0000","last_fullsized":"2024-09-17T04:33:22.036524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:19:22.607029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.676940+0000","last_change":"2024-09-17T04:33:18.690509+0000","last_active":"2024-09-17T04:33:21.676940+0000","last_peered":"2024-09-17T04:33:21.676940+0000","last_clean":"2024-09-17T04:33:21.676940+0000","last_became_active":"2024-09-17T04:33:18.690113+0000","last_became_peered":"2024-09-17T04:33:18.690113+0000","last_unstale":"2024-09-17T04:33:21.676940+0000","last_undegraded":"2024-09-17T04:33:21.676940+0000","last_fullsized":"2024-09-17T04:33:21.676940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:14.180879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035866+0000","last_change":"2024-09-17T04:33:18.691233+0000","last_active":"2024-09-17T04:33:22.035866+0000","last_peered":"2024-09-17T04:33:22.035866+0000","last_clean":"2024-09-17T04:33:22.035866+0000","last_became_active":"2024-09-17T04:33:18.690790+0000","last_became_peered":"2024-09-17T04:33:18.690790+0000","last_unstale":"2024-09-17T04:33:22.035866+0000","last_undegraded":"2024-09-17T04:33:22.035866+0000","last_fullsized":"2024-09-17T04:33:22.035866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:14.180879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035122+0000","last_change":"2024-09-17T04:33:19.695559+0000","last_active":"2024-09-17T04:33:22.035122+0000","last_peered":"2024-09-17T04:33:22.035122+0000","last_clean":"2024-09-17T04:33:22.035122+0000","last_became_active":"2024-09-17T04:33:19.695236+0000","last_became_peered":"2024-09-17T04:33:19.695236+0000","last_unstale":"2024-09-17T04:33:22.035122+0000","last_undegraded":"2024-09-17T04:33:22.035122+0000","last_fullsized":"2024-09-17T04:33:22.035122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:19:59.773350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930110+0000","last_change":"2024-09-17T04:33:18.689567+0000","last_active":"2024-09-17T04:33:20.930110+0000","last_peered":"2024-09-17T04:33:20.930110+0000","last_clean":"2024-09-17T04:33:20.930110+0000","last_became_active":"2024-09-17T04:33:18.689202+0000","last_became_peered":"2024-09-17T04:33:18.689202+0000","last_unstale":"2024-09-17T04:33:20.930110+0000","last_undegraded":"2024-09-17T04:33:20.930110+0000","last_fullsized":"2024-09-17T04:33:20.930110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:23.445663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674031+0000","last_change":"2024-09-17T04:33:19.687275+0000","last_active":"2024-09-17T04:33:21.674031+0000","last_peered":"2024-09-17T04:33:21.674031+0000","last_clean":"2024-09-17T04:33:21.674031+0000","last_became_active":"2024-09-17T04:33:19.684741+0000","last_became_peered":"2024-09-17T04:33:19.684741+0000","last_unstale":"2024-09-17T04:33:21.674031+0000","last_undegraded":"2024-09-17T04:33:21.674031+0000","last_fullsized":"2024-09-17T04:33:21.674031+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:54:01.898864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677365+0000","last_change":"2024-09-17T04:33:18.694470+0000","last_active":"2024-09-17T04:33:21.677365+0000","last_peered":"2024-09-17T04:33:21.677365+0000","last_clean":"2024-09-17T04:33:21.677365+0000","last_became_active":"2024-09-17T04:33:18.694081+0000","last_became_peered":"2024-09-17T04:33:18.694081+0000","last_unstale":"2024-09-17T04:33:21.677365+0000","last_undegraded":"2024-09-17T04:33:21.677365+0000","last_fullsized":"2024-09-17T04:33:21.677365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:27.606246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930243+0000","last_change":"2024-09-17T04:33:19.695793+0000","last_active":"2024-09-17T04:33:20.930243+0000","last_peered":"2024-09-17T04:33:20.930243+0000","last_clean":"2024-09-17T04:33:20.930243+0000","last_became_active":"2024-09-17T04:33:19.695495+0000","last_became_peered":"2024-09-17T04:33:19.695495+0000","last_unstale":"2024-09-17T04:33:20.930243+0000","last_undegraded":"2024-09-17T04:33:20.930243+0000","last_fullsized":"2024-09-17T04:33:20.930243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:10:43.089799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930373+0000","last_change":"2024-09-17T04:33:18.694128+0000","last_active":"2024-09-17T04:33:20.930373+0000","last_peered":"2024-09-17T04:33:20.930373+0000","last_clean":"2024-09-17T04:33:20.930373+0000","last_became_active":"2024-09-17T04:33:18.693766+0000","last_became_peered":"2024-09-17T04:33:18.693766+0000","last_unstale":"2024-09-17T04:33:20.930373+0000","last_undegraded":"2024-09-17T04:33:20.930373+0000","last_fullsized":"2024-09-17T04:33:20.930373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:45:37.943381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036199+0000","last_change":"2024-09-17T04:33:19.691619+0000","last_active":"2024-09-17T04:33:22.036199+0000","last_peered":"2024-09-17T04:33:22.036199+0000","last_clean":"2024-09-17T04:33:22.036199+0000","last_became_active":"2024-09-17T04:33:19.690956+0000","last_became_peered":"2024-09-17T04:33:19.690956+0000","last_unstale":"2024-09-17T04:33:22.036199+0000","last_undegraded":"2024-09-17T04:33:22.036199+0000","last_fullsized":"2024-09-17T04:33:22.036199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:54:01.898864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674190+0000","last_change":"2024-09-17T04:33:18.691537+0000","last_active":"2024-09-17T04:33:21.674190+0000","last_peered":"2024-09-17T04:33:21.674190+0000","last_clean":"2024-09-17T04:33:21.674190+0000","last_became_active":"2024-09-17T04:33:18.691093+0000","last_became_peered":"2024-09-17T04:33:18.691093+0000","last_unstale":"2024-09-17T04:33:21.674190+0000","last_undegraded":"2024-09-17T04:33:21.674190+0000","last_fullsized":"2024-09-17T04:33:21.674190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:27.606246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691734+0000","last_change":"2024-09-17T04:33:19.691142+0000","last_active":"2024-09-17T04:33:20.691734+0000","last_peered":"2024-09-17T04:33:20.691734+0000","last_clean":"2024-09-17T04:33:20.691734+0000","last_became_active":"2024-09-17T04:33:19.690836+0000","last_became_peered":"2024-09-17T04:33:19.690836+0000","last_unstale":"2024-09-17T04:33:20.691734+0000","last_undegraded":"2024-09-17T04:33:20.691734+0000","last_fullsized":"2024-09-17T04:33:20.691734+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:36:40.026116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677278+0000","last_change":"2024-09-17T04:33:18.691686+0000","last_active":"2024-09-17T04:33:21.677278+0000","last_peered":"2024-09-17T04:33:21.677278+0000","last_clean":"2024-09-17T04:33:21.677278+0000","last_became_active":"2024-09-17T04:33:18.690643+0000","last_became_peered":"2024-09-17T04:33:18.690643+0000","last_unstale":"2024-09-17T04:33:21.677278+0000","last_undegraded":"2024-09-17T04:33:21.677278+0000","last_fullsized":"2024-09-17T04:33:21.677278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:53:28.012165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690245+0000","last_change":"2024-09-17T04:33:19.695036+0000","last_active":"2024-09-17T04:33:20.690245+0000","last_peered":"2024-09-17T04:33:20.690245+0000","last_clean":"2024-09-17T04:33:20.690245+0000","last_became_active":"2024-09-17T04:33:19.694670+0000","last_became_peered":"2024-09-17T04:33:19.694670+0000","last_unstale":"2024-09-17T04:33:20.690245+0000","last_undegraded":"2024-09-17T04:33:20.690245+0000","last_fullsized":"2024-09-17T04:33:20.690245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:44.199782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036569+0000","last_change":"2024-09-17T04:33:18.678408+0000","last_active":"2024-09-17T04:33:22.036569+0000","last_peered":"2024-09-17T04:33:22.036569+0000","last_clean":"2024-09-17T04:33:22.036569+0000","last_became_active":"2024-09-17T04:33:18.678054+0000","last_became_peered":"2024-09-17T04:33:18.678054+0000","last_unstale":"2024-09-17T04:33:22.036569+0000","last_undegraded":"2024-09-17T04:33:22.036569+0000","last_fullsized":"2024-09-17T04:33:22.036569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:51.663061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036608+0000","last_change":"2024-09-17T04:33:19.691672+0000","last_active":"2024-09-17T04:33:22.036608+0000","last_peered":"2024-09-17T04:33:22.036608+0000","last_clean":"2024-09-17T04:33:22.036608+0000","last_became_active":"2024-09-17T04:33:19.691319+0000","last_became_peered":"2024-09-17T04:33:19.691319+0000","last_unstale":"2024-09-17T04:33:22.036608+0000","last_undegraded":"2024-09-17T04:33:22.036608+0000","last_fullsized":"2024-09-17T04:33:22.036608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:44.305489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930312+0000","last_change":"2024-09-17T04:33:18.691730+0000","last_active":"2024-09-17T04:33:20.930312+0000","last_peered":"2024-09-17T04:33:20.930312+0000","last_clean":"2024-09-17T04:33:20.930312+0000","last_became_active":"2024-09-17T04:33:18.691046+0000","last_became_peered":"2024-09-17T04:33:18.691046+0000","last_unstale":"2024-09-17T04:33:20.930312+0000","last_undegraded":"2024-09-17T04:33:20.930312+0000","last_fullsized":"2024-09-17T04:33:20.930312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:53:28.012165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690746+0000","last_change":"2024-09-17T04:33:19.699326+0000","last_active":"2024-09-17T04:33:20.690746+0000","last_peered":"2024-09-17T04:33:20.690746+0000","last_clean":"2024-09-17T04:33:20.690746+0000","last_became_active":"2024-09-17T04:33:19.698549+0000","last_became_peered":"2024-09-17T04:33:19.698549+0000","last_unstale":"2024-09-17T04:33:20.690746+0000","last_undegraded":"2024-09-17T04:33:20.690746+0000","last_fullsized":"2024-09-17T04:33:20.690746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:28.772498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.676878+0000","last_change":"2024-09-17T04:33:18.691268+0000","last_active":"2024-09-17T04:33:21.676878+0000","last_peered":"2024-09-17T04:33:21.676878+0000","last_clean":"2024-09-17T04:33:21.676878+0000","last_became_active":"2024-09-17T04:33:18.690946+0000","last_became_peered":"2024-09-17T04:33:18.690946+0000","last_unstale":"2024-09-17T04:33:21.676878+0000","last_undegraded":"2024-09-17T04:33:21.676878+0000","last_fullsized":"2024-09-17T04:33:21.676878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:23.445663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677083+0000","last_change":"2024-09-17T04:33:19.695301+0000","last_active":"2024-09-17T04:33:21.677083+0000","last_peered":"2024-09-17T04:33:21.677083+0000","last_clean":"2024-09-17T04:33:21.677083+0000","last_became_active":"2024-09-17T04:33:19.694951+0000","last_became_peered":"2024-09-17T04:33:19.694951+0000","last_unstale":"2024-09-17T04:33:21.677083+0000","last_undegraded":"2024-09-17T04:33:21.677083+0000","last_fullsized":"2024-09-17T04:33:21.677083+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:22.696900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.683934+0000","last_change":"2024-09-17T04:33:18.684789+0000","last_active":"2024-09-17T04:33:20.683934+0000","last_peered":"2024-09-17T04:33:20.683934+0000","last_clean":"2024-09-17T04:33:20.683934+0000","last_became_active":"2024-09-17T04:33:18.684452+0000","last_became_peered":"2024-09-17T04:33:18.684452+0000","last_unstale":"2024-09-17T04:33:20.683934+0000","last_undegraded":"2024-09-17T04:33:20.683934+0000","last_fullsized":"2024-09-17T04:33:20.683934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:15:32.568929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690027+0000","last_change":"2024-09-17T04:33:19.699323+0000","last_active":"2024-09-17T04:33:20.690027+0000","last_peered":"2024-09-17T04:33:20.690027+0000","last_clean":"2024-09-17T04:33:20.690027+0000","last_became_active":"2024-09-17T04:33:19.698603+0000","last_became_peered":"2024-09-17T04:33:19.698603+0000","last_unstale":"2024-09-17T04:33:20.690027+0000","last_undegraded":"2024-09-17T04:33:20.690027+0000","last_fullsized":"2024-09-17T04:33:20.690027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:13.459991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691033+0000","last_change":"2024-09-17T04:33:18.681934+0000","last_active":"2024-09-17T04:33:20.691033+0000","last_peered":"2024-09-17T04:33:20.691033+0000","last_clean":"2024-09-17T04:33:20.691033+0000","last_became_active":"2024-09-17T04:33:18.681572+0000","last_became_peered":"2024-09-17T04:33:18.681572+0000","last_unstale":"2024-09-17T04:33:20.691033+0000","last_undegraded":"2024-09-17T04:33:20.691033+0000","last_fullsized":"2024-09-17T04:33:20.691033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:59.885891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691190+0000","last_change":"2024-09-17T04:33:19.699024+0000","last_active":"2024-09-17T04:33:20.691190+0000","last_peered":"2024-09-17T04:33:20.691190+0000","last_clean":"2024-09-17T04:33:20.691190+0000","last_became_active":"2024-09-17T04:33:19.698680+0000","last_became_peered":"2024-09-17T04:33:19.698680+0000","last_unstale":"2024-09-17T04:33:20.691190+0000","last_undegraded":"2024-09-17T04:33:20.691190+0000","last_fullsized":"2024-09-17T04:33:20.691190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:29:25.361534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.684435+0000","last_change":"2024-09-17T04:33:18.686803+0000","last_active":"2024-09-17T04:33:20.684435+0000","last_peered":"2024-09-17T04:33:20.684435+0000","last_clean":"2024-09-17T04:33:20.684435+0000","last_became_active":"2024-09-17T04:33:18.686293+0000","last_became_peered":"2024-09-17T04:33:18.686293+0000","last_unstale":"2024-09-17T04:33:20.684435+0000","last_undegraded":"2024-09-17T04:33:20.684435+0000","last_fullsized":"2024-09-17T04:33:20.684435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:07:39.351672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691823+0000","last_change":"2024-09-17T04:33:19.700000+0000","last_active":"2024-09-17T04:33:20.691823+0000","last_peered":"2024-09-17T04:33:20.691823+0000","last_clean":"2024-09-17T04:33:20.691823+0000","last_became_active":"2024-09-17T04:33:19.699695+0000","last_became_peered":"2024-09-17T04:33:19.699695+0000","last_unstale":"2024-09-17T04:33:20.691823+0000","last_undegraded":"2024-09-17T04:33:20.691823+0000","last_fullsized":"2024-09-17T04:33:20.691823+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:00.711952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674416+0000","last_change":"2024-09-17T04:33:18.684112+0000","last_active":"2024-09-17T04:33:21.674416+0000","last_peered":"2024-09-17T04:33:21.674416+0000","last_clean":"2024-09-17T04:33:21.674416+0000","last_became_active":"2024-09-17T04:33:18.683623+0000","last_became_peered":"2024-09-17T04:33:18.683623+0000","last_unstale":"2024-09-17T04:33:21.674416+0000","last_undegraded":"2024-09-17T04:33:21.674416+0000","last_fullsized":"2024-09-17T04:33:21.674416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:53.979348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930602+0000","last_change":"2024-09-17T04:33:19.688648+0000","last_active":"2024-09-17T04:33:20.930602+0000","last_peered":"2024-09-17T04:33:20.930602+0000","last_clean":"2024-09-17T04:33:20.930602+0000","last_became_active":"2024-09-17T04:33:19.688294+0000","last_became_peered":"2024-09-17T04:33:19.688294+0000","last_unstale":"2024-09-17T04:33:20.930602+0000","last_undegraded":"2024-09-17T04:33:20.930602+0000","last_fullsized":"2024-09-17T04:33:20.930602+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:26:38.002625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.684314+0000","last_change":"2024-09-17T04:33:18.690158+0000","last_active":"2024-09-17T04:33:20.684314+0000","last_peered":"2024-09-17T04:33:20.684314+0000","last_clean":"2024-09-17T04:33:20.684314+0000","last_became_active":"2024-09-17T04:33:18.689802+0000","last_became_peered":"2024-09-17T04:33:18.689802+0000","last_unstale":"2024-09-17T04:33:20.684314+0000","last_undegraded":"2024-09-17T04:33:20.684314+0000","last_fullsized":"2024-09-17T04:33:20.684314+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:21:44.098443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690295+0000","last_change":"2024-09-17T04:33:19.702563+0000","last_active":"2024-09-17T04:33:20.690295+0000","last_peered":"2024-09-17T04:33:20.690295+0000","last_clean":"2024-09-17T04:33:20.690295+0000","last_became_active":"2024-09-17T04:33:19.702241+0000","last_became_peered":"2024-09-17T04:33:19.702241+0000","last_unstale":"2024-09-17T04:33:20.690295+0000","last_undegraded":"2024-09-17T04:33:20.690295+0000","last_fullsized":"2024-09-17T04:33:20.690295+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:56:26.043321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690019+0000","last_change":"2024-09-17T04:33:09.978742+0000","last_active":"2024-09-17T04:33:20.690019+0000","last_peered":"2024-09-17T04:33:20.690019+0000","last_clean":"2024-09-17T04:33:20.690019+0000","last_became_active":"2024-09-17T04:33:09.977496+0000","last_became_peered":"2024-09-17T04:33:09.977496+0000","last_unstale":"2024-09-17T04:33:20.690019+0000","last_undegraded":"2024-09-17T04:33:20.690019+0000","last_fullsized":"2024-09-17T04:33:20.690019+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:08.930340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:08.930340+0000","last_clean_scrub_stamp":"2024-09-17T04:33:08.930340+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:21:26.566102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930309+0000","last_change":"2024-09-17T04:33:18.691736+0000","last_active":"2024-09-17T04:33:20.930309+0000","last_peered":"2024-09-17T04:33:20.930309+0000","last_clean":"2024-09-17T04:33:20.930309+0000","last_became_active":"2024-09-17T04:33:18.691410+0000","last_became_peered":"2024-09-17T04:33:18.691410+0000","last_unstale":"2024-09-17T04:33:20.930309+0000","last_undegraded":"2024-09-17T04:33:20.930309+0000","last_fullsized":"2024-09-17T04:33:20.930309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:14.180879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035352+0000","last_change":"2024-09-17T04:33:19.701468+0000","last_active":"2024-09-17T04:33:22.035352+0000","last_peered":"2024-09-17T04:33:22.035352+0000","last_clean":"2024-09-17T04:33:22.035352+0000","last_became_active":"2024-09-17T04:33:19.700764+0000","last_became_peered":"2024-09-17T04:33:19.700764+0000","last_unstale":"2024-09-17T04:33:22.035352+0000","last_undegraded":"2024-09-17T04:33:22.035352+0000","last_fullsized":"2024-09-17T04:33:22.035352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:16.814580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.689948+0000","last_change":"2024-09-17T04:33:18.687948+0000","last_active":"2024-09-17T04:33:20.689948+0000","last_peered":"2024-09-17T04:33:20.689948+0000","last_clean":"2024-09-17T04:33:20.689948+0000","last_became_active":"2024-09-17T04:33:18.687432+0000","last_became_peered":"2024-09-17T04:33:18.687432+0000","last_unstale":"2024-09-17T04:33:20.689948+0000","last_undegraded":"2024-09-17T04:33:20.689948+0000","last_fullsized":"2024-09-17T04:33:20.689948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:45:37.943381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035502+0000","last_change":"2024-09-17T04:33:19.687267+0000","last_active":"2024-09-17T04:33:22.035502+0000","last_peered":"2024-09-17T04:33:22.035502+0000","last_clean":"2024-09-17T04:33:22.035502+0000","last_became_active":"2024-09-17T04:33:19.684583+0000","last_became_peered":"2024-09-17T04:33:19.684583+0000","last_unstale":"2024-09-17T04:33:22.035502+0000","last_undegraded":"2024-09-17T04:33:22.035502+0000","last_fullsized":"2024-09-17T04:33:22.035502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:44.326971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.684040+0000","last_change":"2024-09-17T04:33:18.683812+0000","last_active":"2024-09-17T04:33:20.684040+0000","last_peered":"2024-09-17T04:33:20.684040+0000","last_clean":"2024-09-17T04:33:20.684040+0000","last_became_active":"2024-09-17T04:33:18.683253+0000","last_became_peered":"2024-09-17T04:33:18.683253+0000","last_unstale":"2024-09-17T04:33:20.684040+0000","last_undegraded":"2024-09-17T04:33:20.684040+0000","last_fullsized":"2024-09-17T04:33:20.684040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:00:10.198377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674050+0000","last_change":"2024-09-17T04:33:19.688660+0000","last_active":"2024-09-17T04:33:21.674050+0000","last_peered":"2024-09-17T04:33:21.674050+0000","last_clean":"2024-09-17T04:33:21.674050+0000","last_became_active":"2024-09-17T04:33:19.688336+0000","last_became_peered":"2024-09-17T04:33:19.688336+0000","last_unstale":"2024-09-17T04:33:21.674050+0000","last_undegraded":"2024-09-17T04:33:21.674050+0000","last_fullsized":"2024-09-17T04:33:21.674050+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:12.894886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930502+0000","last_change":"2024-09-17T04:33:18.688213+0000","last_active":"2024-09-17T04:33:20.930502+0000","last_peered":"2024-09-17T04:33:20.930502+0000","last_clean":"2024-09-17T04:33:20.930502+0000","last_became_active":"2024-09-17T04:33:18.687839+0000","last_became_peered":"2024-09-17T04:33:18.687839+0000","last_unstale":"2024-09-17T04:33:20.930502+0000","last_undegraded":"2024-09-17T04:33:20.930502+0000","last_fullsized":"2024-09-17T04:33:20.930502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:27.606246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690522+0000","last_change":"2024-09-17T04:33:19.695246+0000","last_active":"2024-09-17T04:33:20.690522+0000","last_peered":"2024-09-17T04:33:20.690522+0000","last_clean":"2024-09-17T04:33:20.690522+0000","last_became_active":"2024-09-17T04:33:19.694929+0000","last_became_peered":"2024-09-17T04:33:19.694929+0000","last_unstale":"2024-09-17T04:33:20.690522+0000","last_undegraded":"2024-09-17T04:33:20.690522+0000","last_fullsized":"2024-09-17T04:33:20.690522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:38:26.907899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674093+0000","last_change":"2024-09-17T04:33:18.682334+0000","last_active":"2024-09-17T04:33:21.674093+0000","last_peered":"2024-09-17T04:33:21.674093+0000","last_clean":"2024-09-17T04:33:21.674093+0000","last_became_active":"2024-09-17T04:33:18.681988+0000","last_became_peered":"2024-09-17T04:33:18.681988+0000","last_unstale":"2024-09-17T04:33:21.674093+0000","last_undegraded":"2024-09-17T04:33:21.674093+0000","last_fullsized":"2024-09-17T04:33:21.674093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:40.963166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930502+0000","last_change":"2024-09-17T04:33:19.694190+0000","last_active":"2024-09-17T04:33:20.930502+0000","last_peered":"2024-09-17T04:33:20.930502+0000","last_clean":"2024-09-17T04:33:20.930502+0000","last_became_active":"2024-09-17T04:33:19.691073+0000","last_became_peered":"2024-09-17T04:33:19.691073+0000","last_unstale":"2024-09-17T04:33:20.930502+0000","last_undegraded":"2024-09-17T04:33:20.930502+0000","last_fullsized":"2024-09-17T04:33:20.930502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:04:44.157056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690492+0000","last_change":"2024-09-17T04:33:18.687984+0000","last_active":"2024-09-17T04:33:20.690492+0000","last_peered":"2024-09-17T04:33:20.690492+0000","last_clean":"2024-09-17T04:33:20.690492+0000","last_became_active":"2024-09-17T04:33:18.687519+0000","last_became_peered":"2024-09-17T04:33:18.687519+0000","last_unstale":"2024-09-17T04:33:20.690492+0000","last_undegraded":"2024-09-17T04:33:20.690492+0000","last_fullsized":"2024-09-17T04:33:20.690492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:45.809989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674114+0000","last_change":"2024-09-17T04:33:19.688598+0000","last_active":"2024-09-17T04:33:21.674114+0000","last_peered":"2024-09-17T04:33:21.674114+0000","last_clean":"2024-09-17T04:33:21.674114+0000","last_became_active":"2024-09-17T04:33:19.688240+0000","last_became_peered":"2024-09-17T04:33:19.688240+0000","last_unstale":"2024-09-17T04:33:21.674114+0000","last_undegraded":"2024-09-17T04:33:21.674114+0000","last_fullsized":"2024-09-17T04:33:21.674114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:39.340136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.676358+0000","last_change":"2024-09-17T04:33:18.693000+0000","last_active":"2024-09-17T04:33:21.676358+0000","last_peered":"2024-09-17T04:33:21.676358+0000","last_clean":"2024-09-17T04:33:21.676358+0000","last_became_active":"2024-09-17T04:33:18.692677+0000","last_became_peered":"2024-09-17T04:33:18.692677+0000","last_unstale":"2024-09-17T04:33:21.676358+0000","last_undegraded":"2024-09-17T04:33:21.676358+0000","last_fullsized":"2024-09-17T04:33:21.676358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:45:37.943381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035886+0000","last_change":"2024-09-17T04:33:19.695017+0000","last_active":"2024-09-17T04:33:22.035886+0000","last_peered":"2024-09-17T04:33:22.035886+0000","last_clean":"2024-09-17T04:33:22.035886+0000","last_became_active":"2024-09-17T04:33:19.694621+0000","last_became_peered":"2024-09-17T04:33:19.694621+0000","last_unstale":"2024-09-17T04:33:22.035886+0000","last_undegraded":"2024-09-17T04:33:22.035886+0000","last_fullsized":"2024-09-17T04:33:22.035886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:31:38.326621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677397+0000","last_change":"2024-09-17T04:33:18.685926+0000","last_active":"2024-09-17T04:33:21.677397+0000","last_peered":"2024-09-17T04:33:21.677397+0000","last_clean":"2024-09-17T04:33:21.677397+0000","last_became_active":"2024-09-17T04:33:18.685598+0000","last_became_peered":"2024-09-17T04:33:18.685598+0000","last_unstale":"2024-09-17T04:33:21.677397+0000","last_undegraded":"2024-09-17T04:33:21.677397+0000","last_fullsized":"2024-09-17T04:33:21.677397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:45:37.943381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035362+0000","last_change":"2024-09-17T04:33:19.681836+0000","last_active":"2024-09-17T04:33:22.035362+0000","last_peered":"2024-09-17T04:33:22.035362+0000","last_clean":"2024-09-17T04:33:22.035362+0000","last_became_active":"2024-09-17T04:33:19.681085+0000","last_became_peered":"2024-09-17T04:33:19.681085+0000","last_unstale":"2024-09-17T04:33:22.035362+0000","last_undegraded":"2024-09-17T04:33:22.035362+0000","last_fullsized":"2024-09-17T04:33:22.035362+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:58.841388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.689553+0000","last_change":"2024-09-17T04:33:18.688337+0000","last_active":"2024-09-17T04:33:20.689553+0000","last_peered":"2024-09-17T04:33:20.689553+0000","last_clean":"2024-09-17T04:33:20.689553+0000","last_became_active":"2024-09-17T04:33:18.687331+0000","last_became_peered":"2024-09-17T04:33:18.687331+0000","last_unstale":"2024-09-17T04:33:20.689553+0000","last_undegraded":"2024-09-17T04:33:20.689553+0000","last_fullsized":"2024-09-17T04:33:20.689553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:27.606246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.691952+0000","last_change":"2024-09-17T04:33:19.690673+0000","last_active":"2024-09-17T04:33:20.691952+0000","last_peered":"2024-09-17T04:33:20.691952+0000","last_clean":"2024-09-17T04:33:20.691952+0000","last_became_active":"2024-09-17T04:33:19.690356+0000","last_became_peered":"2024-09-17T04:33:19.690356+0000","last_unstale":"2024-09-17T04:33:20.691952+0000","last_undegraded":"2024-09-17T04:33:20.691952+0000","last_fullsized":"2024-09-17T04:33:20.691952+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:42.127433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674278+0000","last_change":"2024-09-17T04:33:18.686258+0000","last_active":"2024-09-17T04:33:21.674278+0000","last_peered":"2024-09-17T04:33:21.674278+0000","last_clean":"2024-09-17T04:33:21.674278+0000","last_became_active":"2024-09-17T04:33:18.685943+0000","last_became_peered":"2024-09-17T04:33:18.685943+0000","last_unstale":"2024-09-17T04:33:21.674278+0000","last_undegraded":"2024-09-17T04:33:21.674278+0000","last_fullsized":"2024-09-17T04:33:21.674278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:23.445663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035293+0000","last_change":"2024-09-17T04:33:19.684510+0000","last_active":"2024-09-17T04:33:22.035293+0000","last_peered":"2024-09-17T04:33:22.035293+0000","last_clean":"2024-09-17T04:33:22.035293+0000","last_became_active":"2024-09-17T04:33:19.683614+0000","last_became_peered":"2024-09-17T04:33:19.683614+0000","last_unstale":"2024-09-17T04:33:22.035293+0000","last_undegraded":"2024-09-17T04:33:22.035293+0000","last_fullsized":"2024-09-17T04:33:22.035293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:22.696900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.674344+0000","last_change":"2024-09-17T04:33:18.684287+0000","last_active":"2024-09-17T04:33:21.674344+0000","last_peered":"2024-09-17T04:33:21.674344+0000","last_clean":"2024-09-17T04:33:21.674344+0000","last_became_active":"2024-09-17T04:33:18.683822+0000","last_became_peered":"2024-09-17T04:33:18.683822+0000","last_unstale":"2024-09-17T04:33:21.674344+0000","last_undegraded":"2024-09-17T04:33:21.674344+0000","last_fullsized":"2024-09-17T04:33:21.674344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:53:28.012165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036101+0000","last_change":"2024-09-17T04:33:19.699298+0000","last_active":"2024-09-17T04:33:22.036101+0000","last_peered":"2024-09-17T04:33:22.036101+0000","last_clean":"2024-09-17T04:33:22.036101+0000","last_became_active":"2024-09-17T04:33:19.698925+0000","last_became_peered":"2024-09-17T04:33:19.698925+0000","last_unstale":"2024-09-17T04:33:22.036101+0000","last_undegraded":"2024-09-17T04:33:22.036101+0000","last_fullsized":"2024-09-17T04:33:22.036101+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:13.459991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690689+0000","last_change":"2024-09-17T04:33:18.685941+0000","last_active":"2024-09-17T04:33:20.690689+0000","last_peered":"2024-09-17T04:33:20.690689+0000","last_clean":"2024-09-17T04:33:20.690689+0000","last_became_active":"2024-09-17T04:33:18.685484+0000","last_became_peered":"2024-09-17T04:33:18.685484+0000","last_unstale":"2024-09-17T04:33:20.690689+0000","last_undegraded":"2024-09-17T04:33:20.690689+0000","last_fullsized":"2024-09-17T04:33:20.690689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:40.963166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:21.677177+0000","last_change":"2024-09-17T04:33:19.685032+0000","last_active":"2024-09-17T04:33:21.677177+0000","last_peered":"2024-09-17T04:33:21.677177+0000","last_clean":"2024-09-17T04:33:21.677177+0000","last_became_active":"2024-09-17T04:33:19.683930+0000","last_became_peered":"2024-09-17T04:33:19.683930+0000","last_unstale":"2024-09-17T04:33:21.677177+0000","last_undegraded":"2024-09-17T04:33:21.677177+0000","last_fullsized":"2024-09-17T04:33:21.677177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:16.814580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930467+0000","last_change":"2024-09-17T04:33:19.691762+0000","last_active":"2024-09-17T04:33:20.930467+0000","last_peered":"2024-09-17T04:33:20.930467+0000","last_clean":"2024-09-17T04:33:20.930467+0000","last_became_active":"2024-09-17T04:33:19.690765+0000","last_became_peered":"2024-09-17T04:33:19.690765+0000","last_unstale":"2024-09-17T04:33:20.930467+0000","last_undegraded":"2024-09-17T04:33:20.930467+0000","last_fullsized":"2024-09-17T04:33:20.930467+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:59.693472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.689803+0000","last_change":"2024-09-17T04:33:18.694221+0000","last_active":"2024-09-17T04:33:20.689803+0000","last_peered":"2024-09-17T04:33:20.689803+0000","last_clean":"2024-09-17T04:33:20.689803+0000","last_became_active":"2024-09-17T04:33:18.693906+0000","last_became_peered":"2024-09-17T04:33:18.693906+0000","last_unstale":"2024-09-17T04:33:20.689803+0000","last_undegraded":"2024-09-17T04:33:20.689803+0000","last_fullsized":"2024-09-17T04:33:20.689803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:53.979348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.930455+0000","last_change":"2024-09-17T04:33:19.692952+0000","last_active":"2024-09-17T04:33:20.930455+0000","last_peered":"2024-09-17T04:33:20.930455+0000","last_clean":"2024-09-17T04:33:20.930455+0000","last_became_active":"2024-09-17T04:33:19.692612+0000","last_became_peered":"2024-09-17T04:33:19.692612+0000","last_unstale":"2024-09-17T04:33:20.930455+0000","last_undegraded":"2024-09-17T04:33:20.930455+0000","last_fullsized":"2024-09-17T04:33:20.930455+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:12:48.590039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:20.690377+0000","last_change":"2024-09-17T04:33:18.692471+0000","last_active":"2024-09-17T04:33:20.690377+0000","last_peered":"2024-09-17T04:33:20.690377+0000","last_clean":"2024-09-17T04:33:20.690377+0000","last_became_active":"2024-09-17T04:33:18.692129+0000","last_became_peered":"2024-09-17T04:33:18.692129+0000","last_unstale":"2024-09-17T04:33:20.690377+0000","last_undegraded":"2024-09-17T04:33:20.690377+0000","last_fullsized":"2024-09-17T04:33:20.690377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:14:47.283001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035813+0000","last_change":"2024-09-17T04:33:19.691269+0000","last_active":"2024-09-17T04:33:22.035813+0000","last_peered":"2024-09-17T04:33:22.035813+0000","last_clean":"2024-09-17T04:33:22.035813+0000","last_became_active":"2024-09-17T04:33:19.690548+0000","last_became_peered":"2024-09-17T04:33:19.690548+0000","last_unstale":"2024-09-17T04:33:22.035813+0000","last_undegraded":"2024-09-17T04:33:22.035813+0000","last_fullsized":"2024-09-17T04:33:22.035813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:03.560173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036319+0000","last_change":"2024-09-17T04:33:18.689670+0000","last_active":"2024-09-17T04:33:22.036319+0000","last_peered":"2024-09-17T04:33:22.036319+0000","last_clean":"2024-09-17T04:33:22.036319+0000","last_became_active":"2024-09-17T04:33:18.689277+0000","last_became_peered":"2024-09-17T04:33:18.689277+0000","last_unstale":"2024-09-17T04:33:22.036319+0000","last_undegraded":"2024-09-17T04:33:22.036319+0000","last_fullsized":"2024-09-17T04:33:22.036319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:30.774768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.036234+0000","last_change":"2024-09-17T04:33:19.686603+0000","last_active":"2024-09-17T04:33:22.036234+0000","last_peered":"2024-09-17T04:33:22.036234+0000","last_clean":"2024-09-17T04:33:22.036234+0000","last_became_active":"2024-09-17T04:33:19.686252+0000","last_became_peered":"2024-09-17T04:33:19.686252+0000","last_unstale":"2024-09-17T04:33:22.036234+0000","last_undegraded":"2024-09-17T04:33:22.036234+0000","last_fullsized":"2024-09-17T04:33:22.036234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:18.658909+0000","last_clean_scrub_stamp":"2024-09-17T04:33:18.658909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:44.199782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T04:33:22.035982+0000","last_change":"2024-09-17T04:33:18.683562+0000","last_active":"2024-09-17T04:33:22.035982+0000","last_peered":"2024-09-17T04:33:22.035982+0000","last_clean":"2024-09-17T04:33:22.035982+0000","last_became_active":"2024-09-17T04:33:18.683155+0000","last_became_peered":"2024-09-17T04:33:18.683155+0000","last_unstale":"2024-09-17T04:33:22.035982+0000","last_undegraded":"2024-09-17T04:33:22.035982+0000","last_fullsized":"2024-09-17T04:33:22.035982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:17.653213+0000","last_clean_scrub_stamp":"2024-09-17T04:33:17.653213+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:59:56.848238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542152,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":823296,"data_stored":669858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":80520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":78998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542152,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":237568,"data_stored":79034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542153,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":80520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542152,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":78908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":79068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":669914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:33:29.089 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T04:33:29.089 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-17T04:33:29.089 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-17T04:33:29.090 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T04:33:29.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.241+0000 7fe9ef0b6640 1 -- 172.21.15.37:0/2177687984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9e80cd260 msgr2=0x7fe9e8167210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:29.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.241+0000 7fe9ef0b6640 1 --2- 172.21.15.37:0/2177687984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9e80cd260 0x7fe9e8167210 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0005fd0 tx=0x7fe9e002a8b0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.241+0000 7fe9ef0b6640 1 -- 172.21.15.37:0/2177687984 shutdown_connections 2024-09-17T04:33:29.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.241+0000 7fe9ef0b6640 1 --2- 172.21.15.37:0/2177687984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9e80cd260 0x7fe9e8167210 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.241+0000 7fe9ef0b6640 1 -- 172.21.15.37:0/2177687984 >> 172.21.15.37:0/2177687984 conn(0x7fe9e8071050 msgr2=0x7fe9e8071460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:29.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.241+0000 7fe9ef0b6640 1 -- 172.21.15.37:0/2177687984 shutdown_connections 2024-09-17T04:33:29.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.241+0000 7fe9ef0b6640 1 -- 172.21.15.37:0/2177687984 wait complete. 2024-09-17T04:33:29.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.243+0000 7fe9ef0b6640 1 Processor -- start 2024-09-17T04:33:29.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.243+0000 7fe9ef0b6640 1 -- start start 2024-09-17T04:33:29.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.245+0000 7fe9ef0b6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9e80d1a10 0x7fe9e815b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:29.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.245+0000 7fe9ef0b6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9e815c830 0x7fe9e815dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:29.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.245+0000 7fe9ef0b6640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe9e8158930 0x7fe9e80b74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:29.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.245+0000 7fe9ef0b6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e8167ae0 con 0x7fe9e80d1a10 2024-09-17T04:33:29.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.245+0000 7fe9ef0b6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e8167960 con 0x7fe9e815c830 2024-09-17T04:33:29.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.245+0000 7fe9ef0b6640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fe9e8167c60 con 0x7fe9e8158930 2024-09-17T04:33:29.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.245+0000 7fe9ece52640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9e80d1a10 0x7fe9e815b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:29.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.245+0000 7fe9ece52640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9e80d1a10 0x7fe9e815b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45956/0 (socket says 172.21.15.37:45956) 2024-09-17T04:33:29.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.245+0000 7fe9ece52640 1 -- 172.21.15.37:0/1028459600 learned_addr learned my addr 172.21.15.37:0/1028459600 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:29.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.245+0000 7fe9ed653640 1 --2- 172.21.15.37:0/1028459600 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe9e8158930 0x7fe9e80b74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:29.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.245+0000 7fe9dffff640 1 --2- 172.21.15.37:0/1028459600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9e815c830 0x7fe9e815dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:29.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.245+0000 7fe9ece52640 1 -- 172.21.15.37:0/1028459600 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe9e8158930 msgr2=0x7fe9e80b74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:29.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.245+0000 7fe9ece52640 1 --2- 172.21.15.37:0/1028459600 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe9e8158930 0x7fe9e80b74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.245+0000 7fe9ece52640 1 -- 172.21.15.37:0/1028459600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9e815c830 msgr2=0x7fe9e815dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:29.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.245+0000 7fe9ece52640 1 --2- 172.21.15.37:0/1028459600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9e815c830 0x7fe9e815dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.245+0000 7fe9ece52640 1 -- 172.21.15.37:0/1028459600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e815bc30 con 0x7fe9e80d1a10 2024-09-17T04:33:29.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.245+0000 7fe9dffff640 1 --2- 172.21.15.37:0/1028459600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9e815c830 0x7fe9e815dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:29.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.245+0000 7fe9ece52640 1 --2- 172.21.15.37:0/1028459600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9e80d1a10 0x7fe9e815b650 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400c3d0 tx=0x7fe9e400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:29.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.246+0000 7fe9dd7fa640 1 -- 172.21.15.37:0/1028459600 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4011790 con 0x7fe9e80d1a10 2024-09-17T04:33:29.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.246+0000 7fe9dd7fa640 1 -- 172.21.15.37:0/1028459600 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9e4017070 con 0x7fe9e80d1a10 2024-09-17T04:33:29.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.246+0000 7fe9ef0b6640 1 -- 172.21.15.37:0/1028459600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9e815d700 con 0x7fe9e80d1a10 2024-09-17T04:33:29.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.246+0000 7fe9dd7fa640 1 -- 172.21.15.37:0/1028459600 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4011cc0 con 0x7fe9e80d1a10 2024-09-17T04:33:29.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.246+0000 7fe9ef0b6640 1 -- 172.21.15.37:0/1028459600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9e815d9a0 con 0x7fe9e80d1a10 2024-09-17T04:33:29.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.247+0000 7fe9ef0b6640 1 -- 172.21.15.37:0/1028459600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9e8075390 con 0x7fe9e80d1a10 2024-09-17T04:33:29.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.247+0000 7fe9dd7fa640 1 -- 172.21.15.37:0/1028459600 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe9e401c460 con 0x7fe9e80d1a10 2024-09-17T04:33:29.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.248+0000 7fe9dd7fa640 1 --2- 172.21.15.37:0/1028459600 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe9d4078bc0 0x7fe9d407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:29.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.248+0000 7fe9dd7fa640 1 -- 172.21.15.37:0/1028459600 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe9e409aae0 con 0x7fe9e80d1a10 2024-09-17T04:33:29.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.248+0000 7fe9dffff640 1 --2- 172.21.15.37:0/1028459600 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe9d4078bc0 0x7fe9d407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:29.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.249+0000 7fe9dffff640 1 --2- 172.21.15.37:0/1028459600 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe9d4078bc0 0x7fe9d407b0d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8006fd0 tx=0x7fe9d8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:29.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.252+0000 7fe9dd7fa640 1 -- 172.21.15.37:0/1028459600 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe9e8075390 con 0x7fe9e80d1a10 2024-09-17T04:33:29.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.475+0000 7fe9ef0b6640 1 -- 172.21.15.37:0/1028459600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe9e80d0240 con 0x7fe9e80d1a10 2024-09-17T04:33:29.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.475+0000 7fe9dd7fa640 1 -- 172.21.15.37:0/1028459600 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fe9e815d700 con 0x7fe9e80d1a10 2024-09-17T04:33:29.475 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:29.475 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T04:33:29.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.477+0000 7fe9ef0b6640 1 -- 172.21.15.37:0/1028459600 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe9d4078bc0 msgr2=0x7fe9d407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:29.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.477+0000 7fe9ef0b6640 1 --2- 172.21.15.37:0/1028459600 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe9d4078bc0 0x7fe9d407b0d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8006fd0 tx=0x7fe9d8008040 comp rx=0 tx=0).stop 2024-09-17T04:33:29.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.477+0000 7fe9ef0b6640 1 -- 172.21.15.37:0/1028459600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9e80d1a10 msgr2=0x7fe9e815b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:29.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.477+0000 7fe9ef0b6640 1 --2- 172.21.15.37:0/1028459600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9e80d1a10 0x7fe9e815b650 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400c3d0 tx=0x7fe9e400c890 comp rx=0 tx=0).stop 2024-09-17T04:33:29.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.477+0000 7fe9ef0b6640 1 -- 172.21.15.37:0/1028459600 shutdown_connections 2024-09-17T04:33:29.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.477+0000 7fe9ef0b6640 1 --2- 172.21.15.37:0/1028459600 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe9d4078bc0 0x7fe9d407b0d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.477+0000 7fe9ef0b6640 1 --2- 172.21.15.37:0/1028459600 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe9e8158930 0x7fe9e80b74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.477+0000 7fe9ef0b6640 1 --2- 172.21.15.37:0/1028459600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9e815c830 0x7fe9e815dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.477+0000 7fe9ef0b6640 1 --2- 172.21.15.37:0/1028459600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9e80d1a10 0x7fe9e815b650 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.477+0000 7fe9ef0b6640 1 -- 172.21.15.37:0/1028459600 >> 172.21.15.37:0/1028459600 conn(0x7fe9e8071050 msgr2=0x7fe9e8154820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:29.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.477+0000 7fe9ef0b6640 1 -- 172.21.15.37:0/1028459600 shutdown_connections 2024-09-17T04:33:29.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.477+0000 7fe9ef0b6640 1 -- 172.21.15.37:0/1028459600 wait complete. 2024-09-17T04:33:29.488 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-17T04:33:29.488 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:33:29.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.604+0000 7f8e35382640 1 -- 172.21.15.37:0/814038101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e300b2560 msgr2=0x7f8e300d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:29.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.604+0000 7f8e35382640 1 --2- 172.21.15.37:0/814038101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e300b2560 0x7f8e300d1e20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8e28005fd0 tx=0x7f8e2802a930 comp rx=0 tx=0).stop 2024-09-17T04:33:29.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.604+0000 7f8e35382640 1 -- 172.21.15.37:0/814038101 shutdown_connections 2024-09-17T04:33:29.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.604+0000 7f8e35382640 1 --2- 172.21.15.37:0/814038101 >> v2:172.21.15.92:3300/0 conn(0x7f8e300cfed0 0x7f8e300b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.604+0000 7f8e35382640 1 --2- 172.21.15.37:0/814038101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e300b2560 0x7f8e300d1e20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.604+0000 7f8e35382640 1 -- 172.21.15.37:0/814038101 >> 172.21.15.37:0/814038101 conn(0x7f8e300d1300 msgr2=0x7f8e300d1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:29.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.604+0000 7f8e35382640 1 -- 172.21.15.37:0/814038101 shutdown_connections 2024-09-17T04:33:29.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.604+0000 7f8e35382640 1 -- 172.21.15.37:0/814038101 wait complete. 2024-09-17T04:33:29.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.606+0000 7f8e35382640 1 Processor -- start 2024-09-17T04:33:29.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.606+0000 7f8e35382640 1 -- start start 2024-09-17T04:33:29.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.607+0000 7f8e35382640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8e300b2560 0x7f8e301f45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:29.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.607+0000 7f8e35382640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e300ce210 0x7f8e300cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:29.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.607+0000 7f8e35382640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e300cfed0 0x7f8e30149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:29.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.607+0000 7f8e35382640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8e300b5750 con 0x7f8e300ce210 2024-09-17T04:33:29.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.607+0000 7f8e35382640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8e300b55d0 con 0x7f8e300cfed0 2024-09-17T04:33:29.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.607+0000 7f8e35382640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f8e300b58d0 con 0x7f8e300b2560 2024-09-17T04:33:29.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.608+0000 7f8e2e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e300ce210 0x7f8e300cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:29.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.608+0000 7f8e2e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e300ce210 0x7f8e300cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45976/0 (socket says 172.21.15.37:45976) 2024-09-17T04:33:29.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.608+0000 7f8e2e7fc640 1 -- 172.21.15.37:0/2774644014 learned_addr learned my addr 172.21.15.37:0/2774644014 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:29.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.608+0000 7f8e2effd640 1 --2- 172.21.15.37:0/2774644014 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8e300b2560 0x7f8e301f45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:29.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.608+0000 7f8e2f7fe640 1 --2- 172.21.15.37:0/2774644014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e300cfed0 0x7f8e30149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:29.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.608+0000 7f8e2e7fc640 1 -- 172.21.15.37:0/2774644014 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8e300b2560 msgr2=0x7f8e301f45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:29.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.608+0000 7f8e2e7fc640 1 --2- 172.21.15.37:0/2774644014 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8e300b2560 0x7f8e301f45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.608+0000 7f8e2e7fc640 1 -- 172.21.15.37:0/2774644014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e300cfed0 msgr2=0x7f8e30149040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:29.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.608+0000 7f8e2e7fc640 1 --2- 172.21.15.37:0/2774644014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e300cfed0 0x7f8e30149040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.608+0000 7f8e2e7fc640 1 -- 172.21.15.37:0/2774644014 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3014a530 con 0x7f8e300ce210 2024-09-17T04:33:29.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.608+0000 7f8e2e7fc640 1 --2- 172.21.15.37:0/2774644014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e300ce210 0x7f8e300cb040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800b420 tx=0x7f8e1800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:29.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.608+0000 7f8e277fe640 1 -- 172.21.15.37:0/2774644014 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e18011830 con 0x7f8e300ce210 2024-09-17T04:33:29.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.608+0000 7f8e277fe640 1 -- 172.21.15.37:0/2774644014 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e18016070 con 0x7f8e300ce210 2024-09-17T04:33:29.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.608+0000 7f8e35382640 1 -- 172.21.15.37:0/2774644014 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e3014aa30 con 0x7f8e300ce210 2024-09-17T04:33:29.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.608+0000 7f8e277fe640 1 -- 172.21.15.37:0/2774644014 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e18011d60 con 0x7f8e300ce210 2024-09-17T04:33:29.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.608+0000 7f8e35382640 1 -- 172.21.15.37:0/2774644014 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e301f7610 con 0x7f8e300ce210 2024-09-17T04:33:29.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.609+0000 7f8e35382640 1 -- 172.21.15.37:0/2774644014 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e3014a7a0 con 0x7f8e300ce210 2024-09-17T04:33:29.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.610+0000 7f8e277fe640 1 -- 172.21.15.37:0/2774644014 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8e1801b460 con 0x7f8e300ce210 2024-09-17T04:33:29.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.611+0000 7f8e277fe640 1 --2- 172.21.15.37:0/2774644014 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8e10078b90 0x7f8e1007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:29.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.611+0000 7f8e277fe640 1 -- 172.21.15.37:0/2774644014 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8e1809a6f0 con 0x7f8e300ce210 2024-09-17T04:33:29.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.611+0000 7f8e2effd640 1 --2- 172.21.15.37:0/2774644014 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8e10078b90 0x7f8e1007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:29.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.612+0000 7f8e2effd640 1 --2- 172.21.15.37:0/2774644014 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8e10078b90 0x7f8e1007b0a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8e28035000 tx=0x7f8e28004ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:29.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.616+0000 7f8e277fe640 1 -- 172.21.15.37:0/2774644014 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8e18020020 con 0x7f8e300ce210 2024-09-17T04:33:29.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.841+0000 7f8e35382640 1 -- 172.21.15.37:0/2774644014 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e301f7120 con 0x7f8e300ce210 2024-09-17T04:33:29.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.842+0000 7f8e277fe640 1 -- 172.21.15.37:0/2774644014 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5217 (secure 0 0 0) 0x7f8e180624d0 con 0x7f8e300ce210 2024-09-17T04:33:29.842 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:29.842 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"btime":"2024-09-17T04:33:22:694087+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4387,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/4285801054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4285801054},{"type":"v1","addr":"172.21.15.92:6836","nonce":4285801054}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4401,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/3514084301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3514084301},{"type":"v1","addr":"172.21.15.37:6834","nonce":3514084301}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4404,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3229527106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":3229527106},{"type":"v1","addr":"172.21.15.37:6835","nonce":3229527106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:19.050573+0000","modified":"2024-09-17T04:33:22.076946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4419":{"gid":4419,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2128554253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2128554253},{"type":"v1","addr":"172.21.15.92:6837","nonce":2128554253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4419,"qdb_cluster":[4419]},"id":1}]} 2024-09-17T04:33:29.842 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-09-17T04:33:29.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.844+0000 7f8e35382640 1 -- 172.21.15.37:0/2774644014 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8e10078b90 msgr2=0x7f8e1007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:29.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.844+0000 7f8e35382640 1 --2- 172.21.15.37:0/2774644014 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8e10078b90 0x7f8e1007b0a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8e28035000 tx=0x7f8e28004ab0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.844+0000 7f8e35382640 1 -- 172.21.15.37:0/2774644014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e300ce210 msgr2=0x7f8e300cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:29.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.844+0000 7f8e35382640 1 --2- 172.21.15.37:0/2774644014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e300ce210 0x7f8e300cb040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800b420 tx=0x7f8e1800b8e0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.844+0000 7f8e35382640 1 -- 172.21.15.37:0/2774644014 shutdown_connections 2024-09-17T04:33:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.844+0000 7f8e35382640 1 --2- 172.21.15.37:0/2774644014 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8e10078b90 0x7f8e1007b0a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.844+0000 7f8e35382640 1 --2- 172.21.15.37:0/2774644014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e300cfed0 0x7f8e30149040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.844+0000 7f8e35382640 1 --2- 172.21.15.37:0/2774644014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e300ce210 0x7f8e300cb040 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.844+0000 7f8e35382640 1 --2- 172.21.15.37:0/2774644014 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8e300b2560 0x7f8e301f45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.844+0000 7f8e35382640 1 -- 172.21.15.37:0/2774644014 >> 172.21.15.37:0/2774644014 conn(0x7f8e300d1300 msgr2=0x7f8e30156060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.844+0000 7f8e35382640 1 -- 172.21.15.37:0/2774644014 shutdown_connections 2024-09-17T04:33:29.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.844+0000 7f8e35382640 1 -- 172.21.15.37:0/2774644014 wait complete. 2024-09-17T04:33:29.856 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:33:29.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.971+0000 7f6d2e10b640 1 -- 172.21.15.37:0/960954699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d280d1a10 msgr2=0x7f6d280ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:29.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.971+0000 7f6d2e10b640 1 --2- 172.21.15.37:0/960954699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d280d1a10 0x7f6d280ccb50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c009830 tx=0x7f6d1c02a510 comp rx=0 tx=0).stop 2024-09-17T04:33:29.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.972+0000 7f6d2e10b640 1 -- 172.21.15.37:0/960954699 shutdown_connections 2024-09-17T04:33:29.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.972+0000 7f6d2e10b640 1 --2- 172.21.15.37:0/960954699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d280d1a10 0x7f6d280ccb50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.972+0000 7f6d2e10b640 1 -- 172.21.15.37:0/960954699 >> 172.21.15.37:0/960954699 conn(0x7f6d28071050 msgr2=0x7f6d28071460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:29.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.972+0000 7f6d2e10b640 1 -- 172.21.15.37:0/960954699 shutdown_connections 2024-09-17T04:33:29.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.972+0000 7f6d2e10b640 1 -- 172.21.15.37:0/960954699 wait complete. 2024-09-17T04:33:29.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.973+0000 7f6d2e10b640 1 Processor -- start 2024-09-17T04:33:29.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.974+0000 7f6d2e10b640 1 -- start start 2024-09-17T04:33:29.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.975+0000 7f6d2e10b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d280cd090 0x7f6d280c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:29.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.975+0000 7f6d2e10b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d280d1a10 0x7f6d28152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:29.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.975+0000 7f6d2e10b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6d2815c830 0x7f6d281f6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:29.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.975+0000 7f6d2e10b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6d280b6440 con 0x7f6d280d1a10 2024-09-17T04:33:29.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.975+0000 7f6d2e10b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6d280b62c0 con 0x7f6d280cd090 2024-09-17T04:33:29.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.975+0000 7f6d2e10b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f6d280b65c0 con 0x7f6d2815c830 2024-09-17T04:33:29.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.975+0000 7f6d26ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d280d1a10 0x7f6d28152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:29.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.975+0000 7f6d26ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d280d1a10 0x7f6d28152850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45994/0 (socket says 172.21.15.37:45994) 2024-09-17T04:33:29.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.975+0000 7f6d26ffd640 1 -- 172.21.15.37:0/803699341 learned_addr learned my addr 172.21.15.37:0/803699341 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:29.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.975+0000 7f6d277fe640 1 --2- 172.21.15.37:0/803699341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d280cd090 0x7f6d280c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:29.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.975+0000 7f6d27fff640 1 --2- 172.21.15.37:0/803699341 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6d2815c830 0x7f6d281f6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:29.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.975+0000 7f6d26ffd640 1 -- 172.21.15.37:0/803699341 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6d2815c830 msgr2=0x7f6d281f6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:29.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.975+0000 7f6d26ffd640 1 --2- 172.21.15.37:0/803699341 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6d2815c830 0x7f6d281f6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.975+0000 7f6d26ffd640 1 -- 172.21.15.37:0/803699341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d280cd090 msgr2=0x7f6d280c5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:29.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.975+0000 7f6d26ffd640 1 --2- 172.21.15.37:0/803699341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d280cd090 0x7f6d280c5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:29.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.975+0000 7f6d26ffd640 1 -- 172.21.15.37:0/803699341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d280c6380 con 0x7f6d280d1a10 2024-09-17T04:33:29.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.975+0000 7f6d26ffd640 1 --2- 172.21.15.37:0/803699341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d280d1a10 0x7f6d28152850 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c000c00 tx=0x7f6d1c033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:29.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.976+0000 7f6d0ffff640 1 -- 172.21.15.37:0/803699341 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c031d60 con 0x7f6d280d1a10 2024-09-17T04:33:29.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.976+0000 7f6d0ffff640 1 -- 172.21.15.37:0/803699341 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d1c0330d0 con 0x7f6d280d1a10 2024-09-17T04:33:29.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.976+0000 7f6d2e10b640 1 -- 172.21.15.37:0/803699341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d281554e0 con 0x7f6d280d1a10 2024-09-17T04:33:29.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.976+0000 7f6d0ffff640 1 -- 172.21.15.37:0/803699341 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c033520 con 0x7f6d280d1a10 2024-09-17T04:33:29.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.976+0000 7f6d2e10b640 1 -- 172.21.15.37:0/803699341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d28155aa0 con 0x7f6d280d1a10 2024-09-17T04:33:29.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.977+0000 7f6d2e10b640 1 -- 172.21.15.37:0/803699341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d280ccb50 con 0x7f6d280d1a10 2024-09-17T04:33:29.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.977+0000 7f6d0ffff640 1 -- 172.21.15.37:0/803699341 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6d1c0035f0 con 0x7f6d280d1a10 2024-09-17T04:33:29.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.978+0000 7f6d0ffff640 1 --2- 172.21.15.37:0/803699341 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6d14078bc0 0x7f6d1407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:29.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.978+0000 7f6d0ffff640 1 -- 172.21.15.37:0/803699341 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6d1c0bc120 con 0x7f6d280d1a10 2024-09-17T04:33:29.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.978+0000 7f6d277fe640 1 --2- 172.21.15.37:0/803699341 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6d14078bc0 0x7f6d1407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:29.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.979+0000 7f6d277fe640 1 --2- 172.21.15.37:0/803699341 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6d14078bc0 0x7f6d1407b0d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6d18000f60 tx=0x7f6d18009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:29.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:29.983+0000 7f6d0ffff640 1 -- 172.21.15.37:0/803699341 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6d1c03f030 con 0x7f6d280d1a10 2024-09-17T04:33:30.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.206+0000 7f6d2e10b640 1 -- 172.21.15.37:0/803699341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d2815bf10 con 0x7f6d280d1a10 2024-09-17T04:33:30.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.207+0000 7f6d0ffff640 1 -- 172.21.15.37:0/803699341 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5217 (secure 0 0 0) 0x7f6d1c0845f0 con 0x7f6d280d1a10 2024-09-17T04:33:30.207 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:30.207 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"btime":"2024-09-17T04:33:22:694087+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4387,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/4285801054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4285801054},{"type":"v1","addr":"172.21.15.92:6836","nonce":4285801054}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4401,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/3514084301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3514084301},{"type":"v1","addr":"172.21.15.37:6834","nonce":3514084301}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4404,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3229527106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":3229527106},{"type":"v1","addr":"172.21.15.37:6835","nonce":3229527106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:19.050573+0000","modified":"2024-09-17T04:33:22.076946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4419":{"gid":4419,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2128554253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2128554253},{"type":"v1","addr":"172.21.15.92:6837","nonce":2128554253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4419,"qdb_cluster":[4419]},"id":1}]} 2024-09-17T04:33:30.207 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-09-17T04:33:30.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.209+0000 7f6d2e10b640 1 -- 172.21.15.37:0/803699341 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6d14078bc0 msgr2=0x7f6d1407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:30.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.209+0000 7f6d2e10b640 1 --2- 172.21.15.37:0/803699341 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6d14078bc0 0x7f6d1407b0d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6d18000f60 tx=0x7f6d18009040 comp rx=0 tx=0).stop 2024-09-17T04:33:30.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.209+0000 7f6d2e10b640 1 -- 172.21.15.37:0/803699341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d280d1a10 msgr2=0x7f6d28152850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:30.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.209+0000 7f6d2e10b640 1 --2- 172.21.15.37:0/803699341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d280d1a10 0x7f6d28152850 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c000c00 tx=0x7f6d1c033d60 comp rx=0 tx=0).stop 2024-09-17T04:33:30.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.209+0000 7f6d2e10b640 1 -- 172.21.15.37:0/803699341 shutdown_connections 2024-09-17T04:33:30.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.209+0000 7f6d2e10b640 1 --2- 172.21.15.37:0/803699341 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6d14078bc0 0x7f6d1407b0d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:30.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.209+0000 7f6d2e10b640 1 --2- 172.21.15.37:0/803699341 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6d2815c830 0x7f6d281f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:30.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.209+0000 7f6d2e10b640 1 --2- 172.21.15.37:0/803699341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d280d1a10 0x7f6d28152850 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:30.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.209+0000 7f6d2e10b640 1 --2- 172.21.15.37:0/803699341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d280cd090 0x7f6d280c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:30.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.209+0000 7f6d2e10b640 1 -- 172.21.15.37:0/803699341 >> 172.21.15.37:0/803699341 conn(0x7f6d28071050 msgr2=0x7f6d280b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:30.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.209+0000 7f6d2e10b640 1 -- 172.21.15.37:0/803699341 shutdown_connections 2024-09-17T04:33:30.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.209+0000 7f6d2e10b640 1 -- 172.21.15.37:0/803699341 wait complete. 2024-09-17T04:33:30.220 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:33:30.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.336+0000 7f1c55c9b640 1 -- 172.21.15.37:0/3195255798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c500cef40 msgr2=0x7f1c500b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:30.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.336+0000 7f1c55c9b640 1 --2- 172.21.15.37:0/3195255798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c500cef40 0x7f1c500b5f40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1c44005fd0 tx=0x7f1c4402a5d0 comp rx=0 tx=0).stop 2024-09-17T04:33:30.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.337+0000 7f1c55c9b640 1 -- 172.21.15.37:0/3195255798 shutdown_connections 2024-09-17T04:33:30.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.337+0000 7f1c55c9b640 1 --2- 172.21.15.37:0/3195255798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c500cef40 0x7f1c500b5f40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:30.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.337+0000 7f1c55c9b640 1 --2- 172.21.15.37:0/3195255798 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1c500d3fe0 0x7f1c5015e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:30.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.337+0000 7f1c55c9b640 1 -- 172.21.15.37:0/3195255798 >> 172.21.15.37:0/3195255798 conn(0x7f1c500d44c0 msgr2=0x7f1c500d48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:30.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.337+0000 7f1c55c9b640 1 -- 172.21.15.37:0/3195255798 shutdown_connections 2024-09-17T04:33:30.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.337+0000 7f1c55c9b640 1 -- 172.21.15.37:0/3195255798 wait complete. 2024-09-17T04:33:30.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.338+0000 7f1c55c9b640 1 Processor -- start 2024-09-17T04:33:30.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.339+0000 7f1c55c9b640 1 -- start start 2024-09-17T04:33:30.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.340+0000 7f1c55c9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c500b6480 0x7f1c5014be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:30.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.340+0000 7f1c55c9b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1c500cef40 0x7f1c501ea830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:30.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.340+0000 7f1c55c9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c500c8fb0 0x7f1c501f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:30.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.340+0000 7f1c55c9b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5006e470 con 0x7f1c500c8fb0 2024-09-17T04:33:30.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.340+0000 7f1c55c9b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1c500b3d30 con 0x7f1c500b6480 2024-09-17T04:33:30.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.340+0000 7f1c55c9b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f1c5006e5f0 con 0x7f1c500cef40 2024-09-17T04:33:30.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.340+0000 7f1c4ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c500c8fb0 0x7f1c501f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:30.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.340+0000 7f1c4ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c500c8fb0 0x7f1c501f8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46010/0 (socket says 172.21.15.37:46010) 2024-09-17T04:33:30.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.340+0000 7f1c4ffff640 1 -- 172.21.15.37:0/340245650 learned_addr learned my addr 172.21.15.37:0/340245650 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:30.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.340+0000 7f1c4effd640 1 --2- 172.21.15.37:0/340245650 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1c500cef40 0x7f1c501ea830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:30.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.340+0000 7f1c4f7fe640 1 --2- 172.21.15.37:0/340245650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c500b6480 0x7f1c5014be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:30.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.340+0000 7f1c4ffff640 1 -- 172.21.15.37:0/340245650 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1c500cef40 msgr2=0x7f1c501ea830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:30.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.340+0000 7f1c4ffff640 1 --2- 172.21.15.37:0/340245650 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1c500cef40 0x7f1c501ea830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:30.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.340+0000 7f1c4ffff640 1 -- 172.21.15.37:0/340245650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c500b6480 msgr2=0x7f1c5014be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:30.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.340+0000 7f1c4ffff640 1 --2- 172.21.15.37:0/340245650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c500b6480 0x7f1c5014be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:30.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.340+0000 7f1c4ffff640 1 -- 172.21.15.37:0/340245650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c500c96e0 con 0x7f1c500c8fb0 2024-09-17T04:33:30.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.341+0000 7f1c4ffff640 1 --2- 172.21.15.37:0/340245650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c500c8fb0 0x7f1c501f8c30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1c4800c760 tx=0x7f1c4800cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:30.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.341+0000 7f1c37fff640 1 -- 172.21.15.37:0/340245650 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c48003b70 con 0x7f1c500c8fb0 2024-09-17T04:33:30.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.341+0000 7f1c37fff640 1 -- 172.21.15.37:0/340245650 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c48003d10 con 0x7f1c500c8fb0 2024-09-17T04:33:30.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.341+0000 7f1c37fff640 1 -- 172.21.15.37:0/340245650 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c480052d0 con 0x7f1c500c8fb0 2024-09-17T04:33:30.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.341+0000 7f1c55c9b640 1 -- 172.21.15.37:0/340245650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c500c99c0 con 0x7f1c500c8fb0 2024-09-17T04:33:30.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.341+0000 7f1c55c9b640 1 -- 172.21.15.37:0/340245650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c500c9f80 con 0x7f1c500c8fb0 2024-09-17T04:33:30.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.342+0000 7f1c55c9b640 1 -- 172.21.15.37:0/340245650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c500c9450 con 0x7f1c500c8fb0 2024-09-17T04:33:30.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.343+0000 7f1c37fff640 1 -- 172.21.15.37:0/340245650 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1c48005470 con 0x7f1c500c8fb0 2024-09-17T04:33:30.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.343+0000 7f1c37fff640 1 --2- 172.21.15.37:0/340245650 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1c3c078b90 0x7f1c3c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:30.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.343+0000 7f1c37fff640 1 -- 172.21.15.37:0/340245650 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1c4809a9d0 con 0x7f1c500c8fb0 2024-09-17T04:33:30.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.343+0000 7f1c4f7fe640 1 --2- 172.21.15.37:0/340245650 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1c3c078b90 0x7f1c3c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:30.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.344+0000 7f1c4f7fe640 1 --2- 172.21.15.37:0/340245650 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1c3c078b90 0x7f1c3c07b0a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1c40004590 tx=0x7f1c4000b390 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:30.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.348+0000 7f1c37fff640 1 -- 172.21.15.37:0/340245650 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1c500b6890 con 0x7f1c500c8fb0 2024-09-17T04:33:30.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.534+0000 7f1c55c9b640 1 -- 172.21.15.37:0/340245650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1c5015e030 con 0x7f1c500c8fb0 2024-09-17T04:33:30.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.535+0000 7f1c37fff640 1 -- 172.21.15.37:0/340245650 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16605 (secure 0 0 0) 0x7f1c5015e030 con 0x7f1c500c8fb0 2024-09-17T04:33:30.535 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:30.535 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":22,"fsid":"a908a603-430c-4209-8294-0c3460b35f42","created":"2024-09-17T04:32:54.139221+0000","modified":"2024-09-17T04:33:20.677493+0000","last_up_change":"2024-09-17T04:33:06.922210+0000","last_in_change":"2024-09-17T04:32:57.754118+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:33:08.461907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:33:16.958785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:18.013768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ea4b8d1-0d22-4dd9-b913-9df585ef24ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6809","nonce":2283587593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6811","nonce":2283587593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6815","nonce":2283587593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6813","nonce":2283587593}]},"public_addr":"172.21.15.37:6809/2283587593","cluster_addr":"172.21.15.37:6811/2283587593","heartbeat_back_addr":"172.21.15.37:6815/2283587593","heartbeat_front_addr":"172.21.15.37:6813/2283587593","state":["exists","up"]},{"osd":1,"uuid":"ac745a33-e8ad-4083-9c18-71bf81982b76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6825","nonce":2657955935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6827","nonce":2657955935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6831","nonce":2657955935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6829","nonce":2657955935}]},"public_addr":"172.21.15.37:6825/2657955935","cluster_addr":"172.21.15.37:6827/2657955935","heartbeat_back_addr":"172.21.15.37:6831/2657955935","heartbeat_front_addr":"172.21.15.37:6829/2657955935","state":["exists","up"]},{"osd":2,"uuid":"3c63f3d8-132a-4735-b0bc-d64313a46503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6817","nonce":579720176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6819","nonce":579720176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6823","nonce":579720176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6821","nonce":579720176}]},"public_addr":"172.21.15.37:6817/579720176","cluster_addr":"172.21.15.37:6819/579720176","heartbeat_back_addr":"172.21.15.37:6823/579720176","heartbeat_front_addr":"172.21.15.37:6821/579720176","state":["exists","up"]},{"osd":3,"uuid":"4f8c6ca2-ea1d-4610-9127-52c4dd8bc8e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6801","nonce":809975826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6803","nonce":809975826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6807","nonce":809975826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6805","nonce":809975826}]},"public_addr":"172.21.15.37:6801/809975826","cluster_addr":"172.21.15.37:6803/809975826","heartbeat_back_addr":"172.21.15.37:6807/809975826","heartbeat_front_addr":"172.21.15.37:6805/809975826","state":["exists","up"]},{"osd":4,"uuid":"63d0c4ab-f115-47b4-9611-12a10153473a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6819","nonce":2509785752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6821","nonce":2509785752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6825","nonce":2509785752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6823","nonce":2509785752}]},"public_addr":"172.21.15.92:6819/2509785752","cluster_addr":"172.21.15.92:6821/2509785752","heartbeat_back_addr":"172.21.15.92:6825/2509785752","heartbeat_front_addr":"172.21.15.92:6823/2509785752","state":["exists","up"]},{"osd":5,"uuid":"4c56c04d-0c07-4f76-9d75-6335121310ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6827","nonce":248263832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6829","nonce":248263832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6833","nonce":248263832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6831","nonce":248263832}]},"public_addr":"172.21.15.92:6827/248263832","cluster_addr":"172.21.15.92:6829/248263832","heartbeat_back_addr":"172.21.15.92:6833/248263832","heartbeat_front_addr":"172.21.15.92:6831/248263832","state":["exists","up"]},{"osd":6,"uuid":"b30eae7c-096a-4d24-ac51-96b8b13ecaee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6811","nonce":4078365985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6813","nonce":4078365985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6817","nonce":4078365985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6815","nonce":4078365985}]},"public_addr":"172.21.15.92:6811/4078365985","cluster_addr":"172.21.15.92:6813/4078365985","heartbeat_back_addr":"172.21.15.92:6817/4078365985","heartbeat_front_addr":"172.21.15.92:6815/4078365985","state":["exists","up"]},{"osd":7,"uuid":"62f12540-d374-4a4c-9ff7-63b7304fe77f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6803","nonce":3444787223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6805","nonce":3444787223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6809","nonce":3444787223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6807","nonce":3444787223}]},"public_addr":"172.21.15.92:6803/3444787223","cluster_addr":"172.21.15.92:6805/3444787223","heartbeat_back_addr":"172.21.15.92:6809/3444787223","heartbeat_front_addr":"172.21.15.92:6807/3444787223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.006420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.111135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.103325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.072656+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.200989+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.060455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.774773+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.865778+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:33:30.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.538+0000 7f1c55c9b640 1 -- 172.21.15.37:0/340245650 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1c3c078b90 msgr2=0x7f1c3c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:30.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.538+0000 7f1c55c9b640 1 --2- 172.21.15.37:0/340245650 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1c3c078b90 0x7f1c3c07b0a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1c40004590 tx=0x7f1c4000b390 comp rx=0 tx=0).stop 2024-09-17T04:33:30.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.538+0000 7f1c55c9b640 1 -- 172.21.15.37:0/340245650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c500c8fb0 msgr2=0x7f1c501f8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:30.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.538+0000 7f1c55c9b640 1 --2- 172.21.15.37:0/340245650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c500c8fb0 0x7f1c501f8c30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1c4800c760 tx=0x7f1c4800cb60 comp rx=0 tx=0).stop 2024-09-17T04:33:30.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.538+0000 7f1c55c9b640 1 -- 172.21.15.37:0/340245650 shutdown_connections 2024-09-17T04:33:30.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.538+0000 7f1c55c9b640 1 --2- 172.21.15.37:0/340245650 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1c3c078b90 0x7f1c3c07b0a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:30.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.538+0000 7f1c55c9b640 1 --2- 172.21.15.37:0/340245650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c500c8fb0 0x7f1c501f8c30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:30.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.538+0000 7f1c55c9b640 1 --2- 172.21.15.37:0/340245650 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1c500cef40 0x7f1c501ea830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:30.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.538+0000 7f1c55c9b640 1 --2- 172.21.15.37:0/340245650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c500b6480 0x7f1c5014be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:30.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.538+0000 7f1c55c9b640 1 -- 172.21.15.37:0/340245650 >> 172.21.15.37:0/340245650 conn(0x7f1c500d44c0 msgr2=0x7f1c5006dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:30.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.538+0000 7f1c55c9b640 1 -- 172.21.15.37:0/340245650 shutdown_connections 2024-09-17T04:33:30.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.538+0000 7f1c55c9b640 1 -- 172.21.15.37:0/340245650 wait complete. 2024-09-17T04:33:30.550 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:33:30.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.666+0000 7f3337bb6640 1 -- 172.21.15.37:0/2462346515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f331c003ac0 msgr2=0x7f331c003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:30.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.666+0000 7f3337bb6640 1 --2- 172.21.15.37:0/2462346515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f331c003ac0 0x7f331c003ef0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f332c00c260 tx=0x7f331c005830 comp rx=0 tx=0).stop 2024-09-17T04:33:30.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.666+0000 7f3336153640 1 -- 172.21.15.37:0/2462346515 reap_dead start 2024-09-17T04:33:30.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.666+0000 7f3337bb6640 1 -- 172.21.15.37:0/2462346515 shutdown_connections 2024-09-17T04:33:30.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.666+0000 7f3337bb6640 1 -- 172.21.15.37:0/2462346515 >> 172.21.15.37:0/2462346515 conn(0x7f333009d800 msgr2=0x7f33300d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:30.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.666+0000 7f3337bb6640 1 -- 172.21.15.37:0/2462346515 shutdown_connections 2024-09-17T04:33:30.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.666+0000 7f3337bb6640 1 -- 172.21.15.37:0/2462346515 wait complete. 2024-09-17T04:33:30.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.668+0000 7f3337bb6640 1 Processor -- start 2024-09-17T04:33:30.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.668+0000 7f3337bb6640 1 -- start start 2024-09-17T04:33:30.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.669+0000 7f3337bb6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f331c003ac0 0x7f33301f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:30.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.669+0000 7f3337bb6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33300ce630 0x7f33301e8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:30.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.669+0000 7f3337bb6640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f33300a1580 0x7f33300a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:30.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.669+0000 7f3337bb6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f33300b4820 con 0x7f331c003ac0 2024-09-17T04:33:30.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.669+0000 7f3337bb6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f33300b46a0 con 0x7f33300ce630 2024-09-17T04:33:30.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.669+0000 7f3337bb6640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f33300b49a0 con 0x7f33300a1580 2024-09-17T04:33:30.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.669+0000 7f3335952640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f331c003ac0 0x7f33301f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:30.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.669+0000 7f3335952640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f331c003ac0 0x7f33301f4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46030/0 (socket says 172.21.15.37:46030) 2024-09-17T04:33:30.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.669+0000 7f3335952640 1 -- 172.21.15.37:0/668257340 learned_addr learned my addr 172.21.15.37:0/668257340 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:30.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.670+0000 7f3336153640 1 --2- 172.21.15.37:0/668257340 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f33300a1580 0x7f33300a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:30.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.670+0000 7f3335151640 1 --2- 172.21.15.37:0/668257340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33300ce630 0x7f33301e8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:30.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.670+0000 7f3335952640 1 -- 172.21.15.37:0/668257340 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f33300a1580 msgr2=0x7f33300a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:30.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.670+0000 7f3335952640 1 --2- 172.21.15.37:0/668257340 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f33300a1580 0x7f33300a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:30.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.670+0000 7f3335952640 1 -- 172.21.15.37:0/668257340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33300ce630 msgr2=0x7f33301e8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:30.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.670+0000 7f3335952640 1 --2- 172.21.15.37:0/668257340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33300ce630 0x7f33301e8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:30.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.670+0000 7f3335952640 1 -- 172.21.15.37:0/668257340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33300a1ed0 con 0x7f331c003ac0 2024-09-17T04:33:30.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.670+0000 7f3335952640 1 --2- 172.21.15.37:0/668257340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f331c003ac0 0x7f33301f4f70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f332c02b110 tx=0x7f332c0011a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:30.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.670+0000 7f33267fc640 1 -- 172.21.15.37:0/668257340 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c031040 con 0x7f331c003ac0 2024-09-17T04:33:30.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.670+0000 7f33267fc640 1 -- 172.21.15.37:0/668257340 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f332c0035f0 con 0x7f331c003ac0 2024-09-17T04:33:30.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.670+0000 7f33267fc640 1 -- 172.21.15.37:0/668257340 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c0039e0 con 0x7f331c003ac0 2024-09-17T04:33:30.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.670+0000 7f3337bb6640 1 -- 172.21.15.37:0/668257340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33301f5880 con 0x7f331c003ac0 2024-09-17T04:33:30.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.670+0000 7f3337bb6640 1 -- 172.21.15.37:0/668257340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33301f5e40 con 0x7f331c003ac0 2024-09-17T04:33:30.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.671+0000 7f3337bb6640 1 -- 172.21.15.37:0/668257340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33300d38f0 con 0x7f331c003ac0 2024-09-17T04:33:30.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.672+0000 7f33267fc640 1 -- 172.21.15.37:0/668257340 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f332c034070 con 0x7f331c003ac0 2024-09-17T04:33:30.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.672+0000 7f33267fc640 1 --2- 172.21.15.37:0/668257340 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3314078bc0 0x7f331407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:30.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.672+0000 7f33267fc640 1 -- 172.21.15.37:0/668257340 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f332c0bae30 con 0x7f331c003ac0 2024-09-17T04:33:30.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.673+0000 7f3335151640 1 --2- 172.21.15.37:0/668257340 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3314078bc0 0x7f331407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:30.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.673+0000 7f3335151640 1 --2- 172.21.15.37:0/668257340 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3314078bc0 0x7f331407b0d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f33300c7d10 tx=0x7f3320006b10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:30.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.677+0000 7f33267fc640 1 -- 172.21.15.37:0/668257340 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f33300d38f0 con 0x7f331c003ac0 2024-09-17T04:33:30.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.900+0000 7f3337bb6640 1 -- 172.21.15.37:0/668257340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33300b4f00 con 0x7f331c003ac0 2024-09-17T04:33:30.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.900+0000 7f33267fc640 1 -- 172.21.15.37:0/668257340 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5217 (secure 0 0 0) 0x7f33300a1ed0 con 0x7f331c003ac0 2024-09-17T04:33:30.901 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:30.901 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"btime":"2024-09-17T04:33:22:694087+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4387,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/4285801054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4285801054},{"type":"v1","addr":"172.21.15.92:6836","nonce":4285801054}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4401,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/3514084301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3514084301},{"type":"v1","addr":"172.21.15.37:6834","nonce":3514084301}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4404,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3229527106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":3229527106},{"type":"v1","addr":"172.21.15.37:6835","nonce":3229527106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:19.050573+0000","modified":"2024-09-17T04:33:22.076946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4419":{"gid":4419,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2128554253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2128554253},{"type":"v1","addr":"172.21.15.92:6837","nonce":2128554253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4419,"qdb_cluster":[4419]},"id":1}]} 2024-09-17T04:33:30.901 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-09-17T04:33:30.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.902+0000 7f3337bb6640 1 -- 172.21.15.37:0/668257340 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3314078bc0 msgr2=0x7f331407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:30.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.903+0000 7f3337bb6640 1 --2- 172.21.15.37:0/668257340 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3314078bc0 0x7f331407b0d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f33300c7d10 tx=0x7f3320006b10 comp rx=0 tx=0).stop 2024-09-17T04:33:30.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.903+0000 7f3337bb6640 1 -- 172.21.15.37:0/668257340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f331c003ac0 msgr2=0x7f33301f4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:30.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.903+0000 7f3337bb6640 1 --2- 172.21.15.37:0/668257340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f331c003ac0 0x7f33301f4f70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f332c02b110 tx=0x7f332c0011a0 comp rx=0 tx=0).stop 2024-09-17T04:33:30.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.903+0000 7f3337bb6640 1 -- 172.21.15.37:0/668257340 shutdown_connections 2024-09-17T04:33:30.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.903+0000 7f3337bb6640 1 --2- 172.21.15.37:0/668257340 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3314078bc0 0x7f331407b0d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:30.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.903+0000 7f3337bb6640 1 --2- 172.21.15.37:0/668257340 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f33300a1580 0x7f33300a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:30.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.903+0000 7f3337bb6640 1 --2- 172.21.15.37:0/668257340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33300ce630 0x7f33301e8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:30.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.903+0000 7f3337bb6640 1 --2- 172.21.15.37:0/668257340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f331c003ac0 0x7f33301f4f70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:30.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.903+0000 7f3337bb6640 1 -- 172.21.15.37:0/668257340 >> 172.21.15.37:0/668257340 conn(0x7f333009d800 msgr2=0x7f3330075fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:30.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.903+0000 7f3337bb6640 1 -- 172.21.15.37:0/668257340 shutdown_connections 2024-09-17T04:33:30.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:30.903+0000 7f3337bb6640 1 -- 172.21.15.37:0/668257340 wait complete. 2024-09-17T04:33:31.317 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T04:33:19.050573+0000', 'modified': '2024-09-17T04:33:22.076946+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4419}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4419': {'gid': 4419, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6837/2128554253', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 2128554253}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 2128554253}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4419, 'qdb_cluster': [4419]} 2024-09-17T04:33:31.317 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T04:33:31.317 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-17T04:33:31.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.445+0000 7f17aeff1640 1 -- 172.21.15.37:0/3187691919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1790003b30 msgr2=0x7f1790003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:31.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.445+0000 7f17aeff1640 1 --2- 172.21.15.37:0/3187691919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1790003b30 0x7f1790003fa0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f17a400c310 tx=0x7f1790005830 comp rx=0 tx=0).stop 2024-09-17T04:33:31.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.445+0000 7f17aeff1640 1 -- 172.21.15.37:0/3187691919 shutdown_connections 2024-09-17T04:33:31.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.445+0000 7f17aeff1640 1 --2- 172.21.15.37:0/3187691919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1790003b30 0x7f1790003fa0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.445+0000 7f17aeff1640 1 -- 172.21.15.37:0/3187691919 >> 172.21.15.37:0/3187691919 conn(0x7f17a806de00 msgr2=0x7f17a806e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:31.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.445+0000 7f17aeff1640 1 -- 172.21.15.37:0/3187691919 shutdown_connections 2024-09-17T04:33:31.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.445+0000 7f17aeff1640 1 -- 172.21.15.37:0/3187691919 wait complete. 2024-09-17T04:33:31.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.447+0000 7f17aeff1640 1 Processor -- start 2024-09-17T04:33:31.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.447+0000 7f17aeff1640 1 -- start start 2024-09-17T04:33:31.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.448+0000 7f17aeff1640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1790003b30 0x7f17a8159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:31.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.448+0000 7f17aeff1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17a8153770 0x7f17a815c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:31.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.448+0000 7f17aeff1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17a8158690 0x7f17a815c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:31.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.448+0000 7f17aeff1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f17a80d1790 con 0x7f17a8158690 2024-09-17T04:33:31.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.448+0000 7f17aeff1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f17a80d1610 con 0x7f17a8153770 2024-09-17T04:33:31.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.448+0000 7f17aeff1640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f17a80d1910 con 0x7f1790003b30 2024-09-17T04:33:31.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.449+0000 7f17ad58e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17a8158690 0x7f17a815c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:31.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.449+0000 7f17ad58e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17a8158690 0x7f17a815c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46036/0 (socket says 172.21.15.37:46036) 2024-09-17T04:33:31.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.449+0000 7f17ad58e640 1 -- 172.21.15.37:0/387446711 learned_addr learned my addr 172.21.15.37:0/387446711 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:31.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.449+0000 7f179ffff640 1 --2- 172.21.15.37:0/387446711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17a8153770 0x7f17a815c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:31.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.449+0000 7f17acd8d640 1 --2- 172.21.15.37:0/387446711 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1790003b30 0x7f17a8159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:31.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.449+0000 7f17ad58e640 1 -- 172.21.15.37:0/387446711 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1790003b30 msgr2=0x7f17a8159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:31.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.449+0000 7f17ad58e640 1 --2- 172.21.15.37:0/387446711 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1790003b30 0x7f17a8159580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.449+0000 7f17ad58e640 1 -- 172.21.15.37:0/387446711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17a8153770 msgr2=0x7f17a815c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:31.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.449+0000 7f17ad58e640 1 --2- 172.21.15.37:0/387446711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17a8153770 0x7f17a815c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.449+0000 7f17ad58e640 1 -- 172.21.15.37:0/387446711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a8159ef0 con 0x7f17a8158690 2024-09-17T04:33:31.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.449+0000 7f17ad58e640 1 --2- 172.21.15.37:0/387446711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17a8158690 0x7f17a815c8e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f17a80cf0e0 tx=0x7f17a0006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:31.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.449+0000 7f179d7fa640 1 -- 172.21.15.37:0/387446711 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0011900 con 0x7f17a8158690 2024-09-17T04:33:31.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.449+0000 7f179d7fa640 1 -- 172.21.15.37:0/387446711 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17a0018070 con 0x7f17a8158690 2024-09-17T04:33:31.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.449+0000 7f17aeff1640 1 -- 172.21.15.37:0/387446711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a81578d0 con 0x7f17a8158690 2024-09-17T04:33:31.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.449+0000 7f179d7fa640 1 -- 172.21.15.37:0/387446711 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0011d40 con 0x7f17a8158690 2024-09-17T04:33:31.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.449+0000 7f17aeff1640 1 -- 172.21.15.37:0/387446711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a8157e90 con 0x7f17a8158690 2024-09-17T04:33:31.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.451+0000 7f179d7fa640 1 -- 172.21.15.37:0/387446711 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f17a001d460 con 0x7f17a8158690 2024-09-17T04:33:31.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.452+0000 7f179d7fa640 1 --2- 172.21.15.37:0/387446711 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1794078b90 0x7f179407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:31.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.452+0000 7f179d7fa640 1 -- 172.21.15.37:0/387446711 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f17a009b5e0 con 0x7f17a8158690 2024-09-17T04:33:31.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.452+0000 7f17acd8d640 1 --2- 172.21.15.37:0/387446711 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1794078b90 0x7f179407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:31.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.453+0000 7f17b603f740 1 Processor -- start 2024-09-17T04:33:31.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.453+0000 7f17b603f740 1 -- start start 2024-09-17T04:33:31.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.453+0000 7f17acd8d640 1 --2- 172.21.15.37:0/387446711 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1794078b90 0x7f179407b0a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f17a4033000 tx=0x7f17a4038040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:31.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.454+0000 7f17b603f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55622b5b8270 0x55622b989fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:31.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.454+0000 7f17b603f740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55622b58f3b0 0x55622b98a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:31.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.454+0000 7f17b603f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b597590 0x55622b9a73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:31.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.454+0000 7f17b603f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55622b8d2820 con 0x55622b597590 2024-09-17T04:33:31.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.454+0000 7f17b603f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55622b8c1c50 con 0x55622b5b8270 2024-09-17T04:33:31.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.454+0000 7f17b603f740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x55622b8c5a60 con 0x55622b58f3b0 2024-09-17T04:33:31.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.454+0000 7f17ad58e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b597590 0x55622b9a73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:31.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.454+0000 7f17ad58e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b597590 0x55622b9a73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46044/0 (socket says 172.21.15.37:46044) 2024-09-17T04:33:31.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.454+0000 7f17ad58e640 1 -- 172.21.15.37:0/329274607 learned_addr learned my addr 172.21.15.37:0/329274607 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:31.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.454+0000 7f17acd8d640 1 --2- 172.21.15.37:0/329274607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55622b5b8270 0x55622b989fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:31.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.454+0000 7f179ffff640 1 --2- 172.21.15.37:0/329274607 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55622b58f3b0 0x55622b98a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:31.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.454+0000 7f17ad58e640 1 -- 172.21.15.37:0/329274607 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55622b58f3b0 msgr2=0x55622b98a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:33:31.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.454+0000 7f17ad58e640 1 --2- 172.21.15.37:0/329274607 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55622b58f3b0 0x55622b98a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.454+0000 7f17ad58e640 1 -- 172.21.15.37:0/329274607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55622b5b8270 msgr2=0x55622b989fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:33:31.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.454+0000 7f17ad58e640 1 --2- 172.21.15.37:0/329274607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55622b5b8270 0x55622b989fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.454+0000 7f17ad58e640 1 -- 172.21.15.37:0/329274607 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55622b623850 con 0x55622b597590 2024-09-17T04:33:31.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.455+0000 7f17ad58e640 1 --2- 172.21.15.37:0/329274607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b597590 0x55622b9a73e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f17a0000f80 tx=0x7f17a0062c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230217b230bb96a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:31.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.455+0000 7f178e7fc640 1 -- 172.21.15.37:0/329274607 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0063330 con 0x55622b597590 2024-09-17T04:33:31.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.455+0000 7f178e7fc640 1 -- 172.21.15.37:0/329274607 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17a0067d60 con 0x55622b597590 2024-09-17T04:33:31.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.455+0000 7f178e7fc640 1 -- 172.21.15.37:0/329274607 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0062790 con 0x55622b597590 2024-09-17T04:33:31.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.455+0000 7f17b603f740 1 -- 172.21.15.37:0/329274607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b597590 msgr2=0x55622b9a73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:31.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.455+0000 7f17b603f740 1 --2- 172.21.15.37:0/329274607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b597590 0x55622b9a73e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f17a0000f80 tx=0x7f17a0062c80 comp rx=0 tx=0).stop 2024-09-17T04:33:31.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.455+0000 7f17b603f740 1 -- 172.21.15.37:0/329274607 shutdown_connections 2024-09-17T04:33:31.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.455+0000 7f17b603f740 1 --2- 172.21.15.37:0/329274607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b597590 0x55622b9a73e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.455+0000 7f17b603f740 1 --2- 172.21.15.37:0/329274607 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55622b58f3b0 0x55622b98a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.455+0000 7f17b603f740 1 --2- 172.21.15.37:0/329274607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55622b5b8270 0x55622b989fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.455+0000 7f17b603f740 1 -- 172.21.15.37:0/329274607 >> 172.21.15.37:0/329274607 conn(0x55622b5ac3c0 msgr2=0x55622b960260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:31.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.456+0000 7f17b603f740 1 -- 172.21.15.37:0/329274607 shutdown_connections 2024-09-17T04:33:31.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.456+0000 7f17b603f740 1 -- 172.21.15.37:0/329274607 wait complete. 2024-09-17T04:33:31.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.456+0000 7f17b603f740 1 -- start start 2024-09-17T04:33:31.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.457+0000 7f17b603f740 1 Processor -- start 2024-09-17T04:33:31.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.458+0000 7f17b603f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b58f3b0 0x55622b975b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:31.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.458+0000 7f17b603f740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55622b597590 0x55622b99dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:31.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.458+0000 7f17b603f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55622b5b8270 0x55622b99e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:31.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.458+0000 7f17b603f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55622b632620 con 0x55622b58f3b0 2024-09-17T04:33:31.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.458+0000 7f17b603f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55622b8675f0 con 0x55622b5b8270 2024-09-17T04:33:31.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.458+0000 7f17b603f740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x55622b99bd40 con 0x55622b597590 2024-09-17T04:33:31.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.458+0000 7f17b603f740 10 client.0 _ll_register_callbacks cb 0x55622b86b0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:33:31.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.458+0000 7f17acd8d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b58f3b0 0x55622b975b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:31.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.458+0000 7f17acd8d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b58f3b0 0x55622b975b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46056/0 (socket says 172.21.15.37:46056) 2024-09-17T04:33:31.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.458+0000 7f17acd8d640 1 -- 172.21.15.37:0/96732522 learned_addr learned my addr 172.21.15.37:0/96732522 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:31.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.458+0000 7f179ffff640 1 --2- 172.21.15.37:0/96732522 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55622b597590 0x55622b99dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:31.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.458+0000 7f17ad58e640 1 --2- 172.21.15.37:0/96732522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55622b5b8270 0x55622b99e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:31.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.458+0000 7f17acd8d640 1 -- 172.21.15.37:0/96732522 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55622b597590 msgr2=0x55622b99dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:33:31.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.458+0000 7f17acd8d640 1 --2- 172.21.15.37:0/96732522 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55622b597590 0x55622b99dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.458+0000 7f17acd8d640 1 -- 172.21.15.37:0/96732522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55622b5b8270 msgr2=0x55622b99e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:33:31.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.458+0000 7f17acd8d640 1 --2- 172.21.15.37:0/96732522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55622b5b8270 0x55622b99e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.458+0000 7f17acd8d640 1 -- 172.21.15.37:0/96732522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55622b69a6b0 con 0x55622b58f3b0 2024-09-17T04:33:31.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.458+0000 7f179ffff640 1 --2- 172.21.15.37:0/96732522 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55622b597590 0x55622b99dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:31.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.459+0000 7f17acd8d640 1 --2- 172.21.15.37:0/96732522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b58f3b0 0x55622b975b90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f17a400c400 tx=0x7f17a40019f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44bcc04974f67eda server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:31.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.459+0000 7f17877fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-17T04:33:31.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.459+0000 7f17877fe640 1 -- 172.21.15.37:0/96732522 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4054060 con 0x55622b58f3b0 2024-09-17T04:33:31.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.459+0000 7f17877fe640 1 -- 172.21.15.37:0/96732522 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f17a4003d00 con 0x55622b58f3b0 2024-09-17T04:33:31.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.459+0000 7f17b603f740 1 -- 172.21.15.37:0/96732522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17a0067d60 con 0x55622b58f3b0 2024-09-17T04:33:31.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.459+0000 7f17877fe640 1 -- 172.21.15.37:0/96732522 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f17a4031040 con 0x55622b58f3b0 2024-09-17T04:33:31.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.459+0000 7f17b603f740 10 client.4713 fetch_fsmap learned FSMap version 8 2024-09-17T04:33:31.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.459+0000 7f17b603f740 1 -- 172.21.15.37:0/96732522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55622b9b4e90 con 0x55622b58f3b0 2024-09-17T04:33:31.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.459+0000 7f17877fe640 1 -- 172.21.15.37:0/96732522 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17a0067d60 con 0x55622b58f3b0 2024-09-17T04:33:31.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.459+0000 7f17877fe640 1 -- 172.21.15.37:0/96732522 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f17a402e490 con 0x55622b58f3b0 2024-09-17T04:33:31.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.459+0000 7f17b603f740 10 client.4713 fetch_fsmap finished waiting for FSMap version 8 2024-09-17T04:33:31.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.459+0000 7f17b603f740 10 client.4713 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-17T04:33:31.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.459+0000 7f17b603f740 20 client.4713 populate_metadata read hostname 'smithi037' 2024-09-17T04:33:31.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.459+0000 7f17b603f740 1 --2- 172.21.15.37:0/96732522 >> [v2:172.21.15.92:6835/2128554253,v1:172.21.15.92:6837/2128554253] conn(0x55622b576e40 0x55622b9b7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:31.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.459+0000 7f17b603f740 4 client.4713 mds_command: new command op to 4419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T04:33:31.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.459+0000 7f17b603f740 1 -- 172.21.15.37:0/96732522 --> [v2:172.21.15.92:6835/2128554253,v1:172.21.15.92:6837/2128554253] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55622b8d4770 con 0x55622b576e40 2024-09-17T04:33:31.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.460+0000 7f179ffff640 1 --2- 172.21.15.37:0/96732522 >> [v2:172.21.15.92:6835/2128554253,v1:172.21.15.92:6837/2128554253] conn(0x55622b576e40 0x55622b9b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:31.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.460+0000 7f179ffff640 1 --2- 172.21.15.37:0/96732522 >> [v2:172.21.15.92:6835/2128554253,v1:172.21.15.92:6837/2128554253] conn(0x55622b576e40 0x55622b9b7790 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ae493fb7dce3f80 server_cookie=9286000fecc69677 in_seq=0 out_seq=0 2024-09-17T04:33:31.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.460+0000 7f17877fe640 10 client.4713 ms_handle_connect on v2:172.21.15.92:6835/2128554253 2024-09-17T04:33:31.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.461+0000 7f17877fe640 1 -- 172.21.15.37:0/96732522 <== mds.0 v2:172.21.15.92:6835/2128554253 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55622b8d4770 con 0x55622b576e40 2024-09-17T04:33:31.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.461+0000 7f17877fe640 10 client.4713 handle_command_reply: tid=0 2024-09-17T04:33:31.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.461+0000 7f17b603f740 1 client.4713 shutdown 2024-09-17T04:33:31.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.461+0000 7f17b603f740 1 -- 172.21.15.37:0/96732522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b58f3b0 msgr2=0x55622b975b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:31.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.461+0000 7f17b603f740 1 --2- 172.21.15.37:0/96732522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b58f3b0 0x55622b975b90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f17a400c400 tx=0x7f17a40019f0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.461+0000 7f17b603f740 1 -- 172.21.15.37:0/96732522 shutdown_connections 2024-09-17T04:33:31.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.461+0000 7f17b603f740 1 --2- 172.21.15.37:0/96732522 >> [v2:172.21.15.92:6835/2128554253,v1:172.21.15.92:6837/2128554253] conn(0x55622b576e40 0x55622b9b7790 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.461+0000 7f17b603f740 1 --2- 172.21.15.37:0/96732522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55622b5b8270 0x55622b99e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.461+0000 7f17b603f740 1 --2- 172.21.15.37:0/96732522 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55622b597590 0x55622b99dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.461+0000 7f17877fe640 0 client.4713 ms_handle_reset on v2:172.21.15.92:6835/2128554253 2024-09-17T04:33:31.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.461+0000 7f17b603f740 1 --2- 172.21.15.37:0/96732522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b58f3b0 0x55622b975b90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.461+0000 7f17b603f740 1 -- 172.21.15.37:0/96732522 >> 172.21.15.37:0/96732522 conn(0x55622b5ac3c0 msgr2=0x55622b978940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:31.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.461+0000 7f17b603f740 1 -- 172.21.15.37:0/96732522 shutdown_connections 2024-09-17T04:33:31.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.461+0000 7f17b603f740 1 -- 172.21.15.37:0/96732522 wait complete. 2024-09-17T04:33:31.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.461+0000 7f17b603f740 20 client.4713 trim_cache size 0 max 16384 2024-09-17T04:33:31.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.480+0000 7f17b603f740 1 Processor -- start 2024-09-17T04:33:31.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.480+0000 7f17b603f740 1 -- start start 2024-09-17T04:33:31.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.481+0000 7f17b603f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55622b58f3b0 0x55622b9702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:31.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.481+0000 7f17b603f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b597590 0x55622b972fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:31.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.481+0000 7f17b603f740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55622b5b8270 0x55622b9734f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:31.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.481+0000 7f17b603f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f17a8201290 con 0x55622b597590 2024-09-17T04:33:31.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.481+0000 7f17b603f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f17a80d3d60 con 0x55622b58f3b0 2024-09-17T04:33:31.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.481+0000 7f17b603f740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f17a81557c0 con 0x55622b5b8270 2024-09-17T04:33:31.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.481+0000 7f179ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b597590 0x55622b972fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:31.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.481+0000 7f179ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b597590 0x55622b972fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46066/0 (socket says 172.21.15.37:46066) 2024-09-17T04:33:31.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.481+0000 7f179ffff640 1 -- 172.21.15.37:0/3499708908 learned_addr learned my addr 172.21.15.37:0/3499708908 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:31.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.482+0000 7f179ffff640 1 -- 172.21.15.37:0/3499708908 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55622b5b8270 msgr2=0x55622b9734f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:33:31.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.482+0000 7f179ffff640 1 --2- 172.21.15.37:0/3499708908 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55622b5b8270 0x55622b9734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.482+0000 7f179ffff640 1 -- 172.21.15.37:0/3499708908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55622b58f3b0 msgr2=0x55622b9702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:33:31.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.482+0000 7f179ffff640 1 --2- 172.21.15.37:0/3499708908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55622b58f3b0 0x55622b9702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.482+0000 7f179ffff640 1 -- 172.21.15.37:0/3499708908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55622b987990 con 0x55622b597590 2024-09-17T04:33:31.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.482+0000 7f179ffff640 1 --2- 172.21.15.37:0/3499708908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b597590 0x55622b972fb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1798005780 tx=0x7f1798005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83767f8fa389a05f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:31.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.482+0000 7f178dffb640 1 -- 172.21.15.37:0/3499708908 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1798010510 con 0x55622b597590 2024-09-17T04:33:31.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.482+0000 7f178dffb640 1 -- 172.21.15.37:0/3499708908 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17980106b0 con 0x55622b597590 2024-09-17T04:33:31.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.482+0000 7f178dffb640 1 -- 172.21.15.37:0/3499708908 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1798010ae0 con 0x55622b597590 2024-09-17T04:33:31.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.483+0000 7f17b603f740 1 -- 172.21.15.37:0/3499708908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b597590 msgr2=0x55622b972fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:31.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.483+0000 7f17b603f740 1 --2- 172.21.15.37:0/3499708908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b597590 0x55622b972fb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1798005780 tx=0x7f1798005f90 comp rx=0 tx=0).stop 2024-09-17T04:33:31.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.483+0000 7f17b603f740 1 -- 172.21.15.37:0/3499708908 shutdown_connections 2024-09-17T04:33:31.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.483+0000 7f17b603f740 1 --2- 172.21.15.37:0/3499708908 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55622b5b8270 0x55622b9734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.483+0000 7f17b603f740 1 --2- 172.21.15.37:0/3499708908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b597590 0x55622b972fb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.483+0000 7f17b603f740 1 --2- 172.21.15.37:0/3499708908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55622b58f3b0 0x55622b9702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.483+0000 7f17b603f740 1 -- 172.21.15.37:0/3499708908 >> 172.21.15.37:0/3499708908 conn(0x55622b5ac3c0 msgr2=0x55622b95b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:31.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.483+0000 7f17b603f740 1 -- 172.21.15.37:0/3499708908 shutdown_connections 2024-09-17T04:33:31.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.483+0000 7f17b603f740 1 -- 172.21.15.37:0/3499708908 wait complete. 2024-09-17T04:33:31.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.484+0000 7f17b603f740 1 -- start start 2024-09-17T04:33:31.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.484+0000 7f17b603f740 1 Processor -- start 2024-09-17T04:33:31.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.485+0000 7f17b603f740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55622b58f3b0 0x55622b98a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:31.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.485+0000 7f17b603f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55622b597590 0x55622ba0d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.485+0000 7f17b603f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b5b8270 0x55622ba246d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.485+0000 7f17b603f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55622b9599a0 con 0x55622b5b8270 2024-09-17T04:33:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.485+0000 7f17b603f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55622b99bf30 con 0x55622b597590 2024-09-17T04:33:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.485+0000 7f17b603f740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x55622b9743a0 con 0x55622b58f3b0 2024-09-17T04:33:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.485+0000 7f17b603f740 10 client.0 _ll_register_callbacks cb 0x7f179000c230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:33:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.485+0000 7f17ad58e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b5b8270 0x55622ba246d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.485+0000 7f17ad58e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b5b8270 0x55622ba246d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46080/0 (socket says 172.21.15.37:46080) 2024-09-17T04:33:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.485+0000 7f17ad58e640 1 -- 172.21.15.37:0/3139867374 learned_addr learned my addr 172.21.15.37:0/3139867374 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.485+0000 7f17acd8d640 1 --2- 172.21.15.37:0/3139867374 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55622b58f3b0 0x55622b98a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.485+0000 7f179ffff640 1 --2- 172.21.15.37:0/3139867374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55622b597590 0x55622ba0d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.486+0000 7f17ad58e640 1 -- 172.21.15.37:0/3139867374 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55622b58f3b0 msgr2=0x55622b98a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:33:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.486+0000 7f17ad58e640 1 --2- 172.21.15.37:0/3139867374 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55622b58f3b0 0x55622b98a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.486+0000 7f17ad58e640 1 -- 172.21.15.37:0/3139867374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55622b597590 msgr2=0x55622ba0d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:33:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.486+0000 7f17ad58e640 1 --2- 172.21.15.37:0/3139867374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55622b597590 0x55622ba0d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.486+0000 7f17ad58e640 1 -- 172.21.15.37:0/3139867374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55622b96faf0 con 0x55622b5b8270 2024-09-17T04:33:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.486+0000 7f17ad58e640 1 --2- 172.21.15.37:0/3139867374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b5b8270 0x55622ba246d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f17a0087040 tx=0x7f17a00627a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbad46f00b66b2c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.486+0000 7f17877fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-17T04:33:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.486+0000 7f17877fe640 1 -- 172.21.15.37:0/3139867374 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0067d60 con 0x55622b5b8270 2024-09-17T04:33:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.486+0000 7f17877fe640 1 -- 172.21.15.37:0/3139867374 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f17a0062000 con 0x55622b5b8270 2024-09-17T04:33:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.486+0000 7f17b603f740 1 -- 172.21.15.37:0/3139867374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17980106b0 con 0x55622b5b8270 2024-09-17T04:33:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.486+0000 7f17877fe640 1 -- 172.21.15.37:0/3139867374 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f17a0082050 con 0x55622b5b8270 2024-09-17T04:33:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.486+0000 7f17b603f740 10 client.4719 fetch_fsmap learned FSMap version 8 2024-09-17T04:33:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.486+0000 7f17b603f740 1 -- 172.21.15.37:0/3139867374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55622ba0de00 con 0x55622b5b8270 2024-09-17T04:33:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.486+0000 7f17877fe640 1 -- 172.21.15.37:0/3139867374 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17980106b0 con 0x55622b5b8270 2024-09-17T04:33:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.486+0000 7f17877fe640 1 -- 172.21.15.37:0/3139867374 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f17a0085080 con 0x55622b5b8270 2024-09-17T04:33:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.486+0000 7f17b603f740 10 client.4719 fetch_fsmap finished waiting for FSMap version 8 2024-09-17T04:33:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.486+0000 7f17b603f740 10 client.4719 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-17T04:33:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.486+0000 7f17b603f740 20 client.4719 populate_metadata read hostname 'smithi037' 2024-09-17T04:33:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.486+0000 7f17b603f740 1 --2- 172.21.15.37:0/3139867374 >> [v2:172.21.15.92:6835/2128554253,v1:172.21.15.92:6837/2128554253] conn(0x55622b973210 0x55622b9a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.486+0000 7f17b603f740 4 client.4719 mds_command: new command op to 4419 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T04:33:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.486+0000 7f17b603f740 1 -- 172.21.15.37:0/3139867374 --> [v2:172.21.15.92:6835/2128554253,v1:172.21.15.92:6837/2128554253] -- command(tid 0: {"prefix": "status"}) -- 0x55622b8d0220 con 0x55622b973210 2024-09-17T04:33:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.487+0000 7f17acd8d640 1 --2- 172.21.15.37:0/3139867374 >> [v2:172.21.15.92:6835/2128554253,v1:172.21.15.92:6837/2128554253] conn(0x55622b973210 0x55622b9a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.487+0000 7f17acd8d640 1 --2- 172.21.15.37:0/3139867374 >> [v2:172.21.15.92:6835/2128554253,v1:172.21.15.92:6837/2128554253] conn(0x55622b973210 0x55622b9a3a80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1080e0d6b880d3f server_cookie=c84a6c00facb193a in_seq=0 out_seq=0 2024-09-17T04:33:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.487+0000 7f17877fe640 10 client.4719 ms_handle_connect on v2:172.21.15.92:6835/2128554253 2024-09-17T04:33:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.487+0000 7f17877fe640 1 -- 172.21.15.37:0/3139867374 <== mds.0 v2:172.21.15.92:6835/2128554253 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55622b8d0220 con 0x55622b973210 2024-09-17T04:33:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.487+0000 7f17877fe640 10 client.4719 handle_command_reply: tid=0 2024-09-17T04:33:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.487+0000 7f17b603f740 1 client.4719 shutdown 2024-09-17T04:33:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.487+0000 7f17b603f740 1 -- 172.21.15.37:0/3139867374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b5b8270 msgr2=0x55622ba246d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.487+0000 7f17b603f740 1 --2- 172.21.15.37:0/3139867374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b5b8270 0x55622ba246d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f17a0087040 tx=0x7f17a00627a0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.487+0000 7f17b603f740 1 -- 172.21.15.37:0/3139867374 shutdown_connections 2024-09-17T04:33:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.487+0000 7f17b603f740 1 --2- 172.21.15.37:0/3139867374 >> [v2:172.21.15.92:6835/2128554253,v1:172.21.15.92:6837/2128554253] conn(0x55622b973210 0x55622b9a3a80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.487+0000 7f17b603f740 1 --2- 172.21.15.37:0/3139867374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55622b5b8270 0x55622ba246d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.487+0000 7f17b603f740 1 --2- 172.21.15.37:0/3139867374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55622b597590 0x55622ba0d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.487+0000 7f17877fe640 0 client.4719 ms_handle_reset on v2:172.21.15.92:6835/2128554253 2024-09-17T04:33:31.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.487+0000 7f17b603f740 1 --2- 172.21.15.37:0/3139867374 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55622b58f3b0 0x55622b98a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.487+0000 7f17b603f740 1 -- 172.21.15.37:0/3139867374 >> 172.21.15.37:0/3139867374 conn(0x55622b5ac3c0 msgr2=0x55622b9702a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:31.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.488+0000 7f17b603f740 1 -- 172.21.15.37:0/3139867374 shutdown_connections 2024-09-17T04:33:31.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.488+0000 7f17b603f740 1 -- 172.21.15.37:0/3139867374 wait complete. 2024-09-17T04:33:31.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.488+0000 7f17b603f740 20 client.4719 trim_cache size 0 max 16384 2024-09-17T04:33:31.492 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T04:33:31.493 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "a908a603-430c-4209-8294-0c3460b35f42", 2024-09-17T04:33:31.493 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-09-17T04:33:31.493 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 4419, 2024-09-17T04:33:31.493 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-17T04:33:31.493 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-17T04:33:31.493 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-17T04:33:31.493 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 11.821788986, 2024-09-17T04:33:31.493 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 8, 2024-09-17T04:33:31.493 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 22, 2024-09-17T04:33:31.493 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 22, 2024-09-17T04:33:31.493 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 17.832874836999999 2024-09-17T04:33:31.493 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T04:33:31.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.488+0000 7f17aeff1640 1 -- 172.21.15.37:0/387446711 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1794078b90 msgr2=0x7f179407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:31.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.488+0000 7f17aeff1640 1 --2- 172.21.15.37:0/387446711 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1794078b90 0x7f179407b0a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f17a4033000 tx=0x7f17a4038040 comp rx=0 tx=0).stop 2024-09-17T04:33:31.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.488+0000 7f17aeff1640 1 -- 172.21.15.37:0/387446711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17a8158690 msgr2=0x7f17a815c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:31.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.488+0000 7f17aeff1640 1 --2- 172.21.15.37:0/387446711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17a8158690 0x7f17a815c8e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f17a80cf0e0 tx=0x7f17a0006430 comp rx=0 tx=0).stop 2024-09-17T04:33:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.488+0000 7f17aeff1640 1 -- 172.21.15.37:0/387446711 shutdown_connections 2024-09-17T04:33:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.488+0000 7f17aeff1640 1 --2- 172.21.15.37:0/387446711 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1794078b90 0x7f179407b0a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.488+0000 7f17aeff1640 1 --2- 172.21.15.37:0/387446711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17a8158690 0x7f17a815c8e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.488+0000 7f17aeff1640 1 --2- 172.21.15.37:0/387446711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17a8153770 0x7f17a815c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.488+0000 7f17aeff1640 1 --2- 172.21.15.37:0/387446711 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1790003b30 0x7f17a8159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.488+0000 7f17aeff1640 1 -- 172.21.15.37:0/387446711 >> 172.21.15.37:0/387446711 conn(0x7f17a806de00 msgr2=0x7f17a81616c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.488+0000 7f17aeff1640 1 -- 172.21.15.37:0/387446711 shutdown_connections 2024-09-17T04:33:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.488+0000 7f17aeff1640 1 -- 172.21.15.37:0/387446711 wait complete. 2024-09-17T04:33:31.499 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-17T04:33:31.524 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-17T04:33:31.525 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-17T04:33:31.525 INFO:tasks.ceph_fuse:client.0 config is {} 2024-09-17T04:33:31.525 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T04:33:31.525 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T04:33:31.525 INFO:tasks.ceph_fuse:client.2 config is {} 2024-09-17T04:33:31.525 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T04:33:31.525 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-17T04:33:31.525 INFO:tasks.ceph_fuse:client.1 config is {} 2024-09-17T04:33:31.525 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T04:33:31.526 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T04:33:31.526 INFO:tasks.ceph_fuse:client.3 config is {} 2024-09-17T04:33:31.526 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T04:33:31.526 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-17T04:33:31.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:31.526 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-09-17T04:33:31.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:31.542 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-09-17T04:33:31.610 INFO:teuthology.orchestra.run.smithi037.stderr:Cannot find device "ceph-brx" 2024-09-17T04:33:31.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T04:33:31.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:31.613 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2024-09-17T04:33:31.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:31.632 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2024-09-17T04:33:31.704 INFO:teuthology.orchestra.run.smithi092.stderr:Cannot find device "ceph-brx" 2024-09-17T04:33:31.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T04:33:31.706 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-17T04:33:31.706 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-17T04:33:31.707 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:33:31.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.824+0000 7fe91ee4d640 1 -- 172.21.15.37:0/3214540415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9180af1d0 msgr2=0x7fe9180ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:31.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.824+0000 7fe91ee4d640 1 --2- 172.21.15.37:0/3214540415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9180af1d0 0x7fe9180ceac0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe9180afba0 tx=0x7fe90c02a5f0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.824+0000 7fe91ee4d640 1 -- 172.21.15.37:0/3214540415 shutdown_connections 2024-09-17T04:33:31.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.824+0000 7fe91ee4d640 1 --2- 172.21.15.37:0/3214540415 >> v2:172.21.15.92:3300/0 conn(0x7fe918070650 0x7fe9180ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.824+0000 7fe91ee4d640 1 --2- 172.21.15.37:0/3214540415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9180af1d0 0x7fe9180ceac0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.824+0000 7fe91ee4d640 1 -- 172.21.15.37:0/3214540415 >> 172.21.15.37:0/3214540415 conn(0x7fe9180b1ca0 msgr2=0x7fe9180b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:31.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.825+0000 7fe91ee4d640 1 -- 172.21.15.37:0/3214540415 shutdown_connections 2024-09-17T04:33:31.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.825+0000 7fe91ee4d640 1 -- 172.21.15.37:0/3214540415 wait complete. 2024-09-17T04:33:31.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.826+0000 7fe91ee4d640 1 Processor -- start 2024-09-17T04:33:31.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.826+0000 7fe91ee4d640 1 -- start start 2024-09-17T04:33:31.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.828+0000 7fe91ee4d640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe9180af1d0 0x7fe9181e50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:31.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.828+0000 7fe91ee4d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9180b52d0 0x7fe9180c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:31.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.828+0000 7fe91ee4d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9181f8b10 0x7fe91815a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:31.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.828+0000 7fe91ee4d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe9180cd1c0 con 0x7fe9181f8b10 2024-09-17T04:33:31.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.828+0000 7fe91ee4d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe9180cd340 con 0x7fe9180b52d0 2024-09-17T04:33:31.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.828+0000 7fe91ee4d640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fe9180cd4c0 con 0x7fe9180af1d0 2024-09-17T04:33:31.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.828+0000 7fe91d3ea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9181f8b10 0x7fe91815a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:31.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.828+0000 7fe91d3ea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9181f8b10 0x7fe91815a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46098/0 (socket says 172.21.15.37:46098) 2024-09-17T04:33:31.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.828+0000 7fe91d3ea640 1 -- 172.21.15.37:0/1970889533 learned_addr learned my addr 172.21.15.37:0/1970889533 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:31.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.828+0000 7fe917fff640 1 --2- 172.21.15.37:0/1970889533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9180b52d0 0x7fe9180c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:31.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.828+0000 7fe91cbe9640 1 --2- 172.21.15.37:0/1970889533 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe9180af1d0 0x7fe9181e50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:31.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.828+0000 7fe91d3ea640 1 -- 172.21.15.37:0/1970889533 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe9180af1d0 msgr2=0x7fe9181e50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:31.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.828+0000 7fe91d3ea640 1 --2- 172.21.15.37:0/1970889533 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe9180af1d0 0x7fe9181e50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.828+0000 7fe91d3ea640 1 -- 172.21.15.37:0/1970889533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9180b52d0 msgr2=0x7fe9180c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:31.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.828+0000 7fe91d3ea640 1 --2- 172.21.15.37:0/1970889533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9180b52d0 0x7fe9180c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:31.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.828+0000 7fe91d3ea640 1 -- 172.21.15.37:0/1970889533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9181f9240 con 0x7fe9181f8b10 2024-09-17T04:33:31.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.828+0000 7fe91cbe9640 1 --2- 172.21.15.37:0/1970889533 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe9180af1d0 0x7fe9181e50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:31.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.828+0000 7fe917fff640 1 --2- 172.21.15.37:0/1970889533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9180b52d0 0x7fe9180c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:31.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.828+0000 7fe91d3ea640 1 --2- 172.21.15.37:0/1970889533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9181f8b10 0x7fe91815a350 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe91000cad0 tx=0x7fe91000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:31.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.828+0000 7fe9157fa640 1 -- 172.21.15.37:0/1970889533 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91000ae20 con 0x7fe9181f8b10 2024-09-17T04:33:31.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.829+0000 7fe9157fa640 1 -- 172.21.15.37:0/1970889533 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe91000f540 con 0x7fe9181f8b10 2024-09-17T04:33:31.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.829+0000 7fe9157fa640 1 -- 172.21.15.37:0/1970889533 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91000f990 con 0x7fe9181f8b10 2024-09-17T04:33:31.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.829+0000 7fe91ee4d640 1 -- 172.21.15.37:0/1970889533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9181f9520 con 0x7fe9181f8b10 2024-09-17T04:33:31.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.829+0000 7fe91ee4d640 1 -- 172.21.15.37:0/1970889533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9180c85d0 con 0x7fe9181f8b10 2024-09-17T04:33:31.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.830+0000 7fe91ee4d640 1 -- 172.21.15.37:0/1970889533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9180ccc80 con 0x7fe9181f8b10 2024-09-17T04:33:31.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.830+0000 7fe9157fa640 1 -- 172.21.15.37:0/1970889533 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe91000fb30 con 0x7fe9181f8b10 2024-09-17T04:33:31.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.831+0000 7fe9157fa640 1 --2- 172.21.15.37:0/1970889533 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe8fc078bf0 0x7fe8fc07b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:31.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.831+0000 7fe9157fa640 1 -- 172.21.15.37:0/1970889533 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe91009b7c0 con 0x7fe9181f8b10 2024-09-17T04:33:31.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.831+0000 7fe91cbe9640 1 --2- 172.21.15.37:0/1970889533 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe8fc078bf0 0x7fe8fc07b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:31.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.832+0000 7fe91cbe9640 1 --2- 172.21.15.37:0/1970889533 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe8fc078bf0 0x7fe8fc07b100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe90c033000 tx=0x7fe90c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:31.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:31.835+0000 7fe9157fa640 1 -- 172.21.15.37:0/1970889533 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe910014030 con 0x7fe9181f8b10 2024-09-17T04:33:32.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:32.058+0000 7fe91ee4d640 1 -- 172.21.15.37:0/1970889533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe9180b25a0 con 0x7fe9181f8b10 2024-09-17T04:33:32.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:32.058+0000 7fe9157fa640 1 -- 172.21.15.37:0/1970889533 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fe91000fe20 con 0x7fe9181f8b10 2024-09-17T04:33:32.058 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:33:32.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:32.060+0000 7fe91ee4d640 1 -- 172.21.15.37:0/1970889533 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe8fc078bf0 msgr2=0x7fe8fc07b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:32.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:32.060+0000 7fe91ee4d640 1 --2- 172.21.15.37:0/1970889533 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe8fc078bf0 0x7fe8fc07b100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe90c033000 tx=0x7fe90c004730 comp rx=0 tx=0).stop 2024-09-17T04:33:32.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:32.060+0000 7fe91ee4d640 1 -- 172.21.15.37:0/1970889533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9181f8b10 msgr2=0x7fe91815a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:32.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:32.060+0000 7fe91ee4d640 1 --2- 172.21.15.37:0/1970889533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9181f8b10 0x7fe91815a350 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe91000cad0 tx=0x7fe91000cf90 comp rx=0 tx=0).stop 2024-09-17T04:33:32.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:32.060+0000 7fe91ee4d640 1 -- 172.21.15.37:0/1970889533 shutdown_connections 2024-09-17T04:33:32.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:32.060+0000 7fe91ee4d640 1 --2- 172.21.15.37:0/1970889533 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe8fc078bf0 0x7fe8fc07b100 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:32.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:32.060+0000 7fe91ee4d640 1 --2- 172.21.15.37:0/1970889533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9181f8b10 0x7fe91815a350 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:32.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:32.060+0000 7fe91ee4d640 1 --2- 172.21.15.37:0/1970889533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9180b52d0 0x7fe9180c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:32.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:32.060+0000 7fe91ee4d640 1 --2- 172.21.15.37:0/1970889533 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe9180af1d0 0x7fe9181e50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:32.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:32.060+0000 7fe91ee4d640 1 -- 172.21.15.37:0/1970889533 >> 172.21.15.37:0/1970889533 conn(0x7fe9180b1ca0 msgr2=0x7fe91815b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:32.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:32.060+0000 7fe91ee4d640 1 -- 172.21.15.37:0/1970889533 shutdown_connections 2024-09-17T04:33:32.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:32.060+0000 7fe91ee4d640 1 -- 172.21.15.37:0/1970889533 wait complete. 2024-09-17T04:33:32.071 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T04:33:32.071 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:33:32.071 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-09-17T04:33:32.071 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T04:33:32.071 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T04:33:32.071 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T04:33:32.071 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T04:33:32.072 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:33:32.072 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:33:32.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:32.072 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-09-17T04:33:32.088 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:33:32.088 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:33:32.088 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:33:32.088 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:32.088 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-09-17T04:33:32.089 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:32.089 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:33:32.089 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:32.089 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-09-17T04:33:32.089 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:33:32.089 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:32.089 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-09-17T04:33:32.089 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:33:32.089 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:32.089 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-09-17T04:33:32.089 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-17T04:33:32.089 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42740sec preferred_lft 42740sec 2024-09-17T04:33:32.089 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-09-17T04:33:32.089 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:32.090 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:33:32.090 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:32.090 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-09-17T04:33:32.090 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-17T04:33:32.090 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:33:32.090 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-17T04:33:32.090 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add name ceph-brx type bridge 2024-09-17T04:33:32.090 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr flush dev ceph-brx 2024-09-17T04:33:32.091 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set ceph-brx up 2024-09-17T04:33:32.091 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-17T04:33:32.091 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-17T04:33:32.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:33:32.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:33:32.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:32.294 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:33:32.319 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-09-17T04:33:32.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:32.321 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-09-17T04:33:32.392 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-09-17T04:33:32.392 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:33:32.393 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-17T04:33:32.393 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-17T04:33:32.393 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:33:32.393 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:33:32.393 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-17T04:33:32.393 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T04:33:32.393 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T04:33:32.393 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T04:33:32.393 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-17T04:33:32.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:33:32.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:33:32.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:32.691 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-09-17T04:33:32.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:32.707 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list-id 2024-09-17T04:33:32.764 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:33:32.764 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-17T04:33:32.764 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:33:32.764 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-17T04:33:32.764 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-17T04:33:32.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:33:32.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:33:32.884 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-17T04:33:32.884 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:33:32.884 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-17T04:33:32.884 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-17T04:33:32.884 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-17T04:33:32.885 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-17T04:33:32.885 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-17T04:33:32.885 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-17T04:33:32.885 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-17T04:33:32.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:33:33.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:33:33.027 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:33:33.027 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-17T04:33:33.028 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-09-17T04:33:33.028 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set dev brx.0 master ceph-brx 2024-09-17T04:33:33.028 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-17T04:33:33.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:33:33.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:33:33.108 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-17T04:33:33.108 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:33:33.108 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T04:33:33.124 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:33:33.124 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:33:33.125 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:33:33.181 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-09-17T04:33:33.248 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T04:33:33.305 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-09-17T04:33:33.305 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-09-17T04:33:33.306 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-09-17T04:33:33.306 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-09-17T04:33:33.306 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-09-17T04:33:33.306 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-09-17T04:33:33.306 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-09-17T04:33:33.306 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-09-17T04:33:33.306 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-09-17T04:33:33.306 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-09-17T04:33:33.307 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-09-17T04:33:33.307 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/selinux 2024-09-17T04:33:33.307 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T04:33:33.307 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-09-17T04:33:33.307 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-09-17T04:33:33.307 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-09-17T04:33:33.307 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-09-17T04:33:33.307 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-09-17T04:33:33.308 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-09-17T04:33:33.308 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T04:33:33.308 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T04:33:33.308 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T04:33:33.308 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T04:33:33.308 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-09-17T04:33:33.308 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-09-17T04:33:33.308 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T04:33:33.308 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T04:33:33.308 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T04:33:33.308 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T04:33:33.308 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-09-17T04:33:33.308 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:33:33.309 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:33:33.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:33.309 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-09-17T04:33:33.361 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T04:33:33.362 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-17T04:33:33.404 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-09-17T04:33:33.430 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T04:33:33.489 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-09-17T04:33:33.489 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-09-17T04:33:33.489 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-09-17T04:33:33.489 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-09-17T04:33:33.489 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-09-17T04:33:33.489 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-09-17T04:33:33.490 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-09-17T04:33:33.490 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-09-17T04:33:33.490 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-09-17T04:33:33.490 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-09-17T04:33:33.490 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-09-17T04:33:33.490 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/selinux 2024-09-17T04:33:33.490 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T04:33:33.490 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-09-17T04:33:33.490 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-09-17T04:33:33.490 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-09-17T04:33:33.490 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-09-17T04:33:33.490 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-09-17T04:33:33.490 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-09-17T04:33:33.490 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T04:33:33.491 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T04:33:33.491 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T04:33:33.491 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T04:33:33.491 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-09-17T04:33:33.491 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-09-17T04:33:33.491 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T04:33:33.491 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T04:33:33.491 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T04:33:33.491 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T04:33:33.491 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-09-17T04:33:33.491 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:33:33.491 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:33:33.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:33.492 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-09-17T04:33:33.505 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:2024-09-17T04:33:33.505+0000 7f252ce17480 -1 init, newargv = 0x557a95b79990 newargc=15 2024-09-17T04:33:33.505 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[45322]: starting ceph client 2024-09-17T04:33:33.510 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[45322]: starting fuse 2024-09-17T04:33:33.521 INFO:teuthology.orchestra.run.smithi037.stdout:42 2024-09-17T04:33:33.522 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-17T04:33:33.522 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-09-17T04:33:33.522 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-09-17T04:33:33.522 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-09-17T04:33:33.522 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-17T04:33:33.522 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-09-17T04:33:33.522 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-17T04:33:33.522 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-09-17T04:33:33.522 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-17T04:33:33.522 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-09-17T04:33:33.523 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-17T04:33:33.523 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-17T04:33:33.523 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-09-17T04:33:33.523 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-09-17T04:33:33.523 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-09-17T04:33:33.523 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-09-17T04:33:33.523 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-17T04:33:33.523 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-09-17T04:33:33.523 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T04:33:33.523 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T04:33:33.523 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T04:33:33.524 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-09-17T04:33:33.524 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-09-17T04:33:33.524 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-09-17T04:33:33.524 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T04:33:33.524 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-17T04:33:33.524 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.0")) 2024-09-17T04:33:33.524 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-17T04:33:33.642 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.0.45322.asok 2024-09-17T04:33:33.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:33:33.653 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.45322.asok 2024-09-17T04:33:33.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:33.653 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.45322.asok status 2024-09-17T04:33:33.799 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T04:33:33.799 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-09-17T04:33:33.800 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T04:33:33.800 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T04:33:33.800 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "0", 2024-09-17T04:33:33.800 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-09-17T04:33:33.800 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-17T04:33:33.801 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "45322", 2024-09-17T04:33:33.801 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-09-17T04:33:33.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:33.801 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-09-17T04:33:33.801 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-09-17T04:33:33.801 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 4734, 2024-09-17T04:33:33.802 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-09-17T04:33:33.802 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-09-17T04:33:33.802 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-09-17T04:33:33.802 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 4734 2024-09-17T04:33:33.802 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:33.802 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-09-17T04:33:33.803 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-09-17T04:33:33.803 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-09-17T04:33:33.803 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 315510129 2024-09-17T04:33:33.803 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:33.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:33.803 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-09-17T04:33:33.804 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-09-17T04:33:33.804 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-09-17T04:33:33.804 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 315510129 2024-09-17T04:33:33.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:33.804 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.4734 192.168.0.1:0/315510129", 2024-09-17T04:33:33.804 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.1:0/315510129", 2024-09-17T04:33:33.805 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-09-17T04:33:33.805 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 8, 2024-09-17T04:33:33.805 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 22, 2024-09-17T04:33:33.805 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-09-17T04:33:33.805 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-09-17T04:33:33.805 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-09-17T04:33:33.806 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T04:33:33.809 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-17T04:33:33.810 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:33:33.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.927+0000 7f9165cbd640 1 -- 172.21.15.37:0/1385729714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f916009dc60 msgr2=0x7f91600ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:33.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.927+0000 7f9165cbd640 1 --2- 172.21.15.37:0/1385729714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f916009dc60 0x7f91600ce740 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9150009830 tx=0x7f915002a5c0 comp rx=0 tx=0).stop 2024-09-17T04:33:33.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.927+0000 7f9165cbd640 1 -- 172.21.15.37:0/1385729714 shutdown_connections 2024-09-17T04:33:33.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.927+0000 7f9165cbd640 1 --2- 172.21.15.37:0/1385729714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f916009dc60 0x7f91600ce740 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:33.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.927+0000 7f9165cbd640 1 -- 172.21.15.37:0/1385729714 >> 172.21.15.37:0/1385729714 conn(0x7f91600b1ca0 msgr2=0x7f91600b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:33.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.927+0000 7f9165cbd640 1 -- 172.21.15.37:0/1385729714 shutdown_connections 2024-09-17T04:33:33.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.927+0000 7f9165cbd640 1 -- 172.21.15.37:0/1385729714 wait complete. 2024-09-17T04:33:33.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.929+0000 7f9165cbd640 1 Processor -- start 2024-09-17T04:33:33.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.929+0000 7f9165cbd640 1 -- start start 2024-09-17T04:33:33.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.930+0000 7f9165cbd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f916006ee30 0x7f916009c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:33.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.930+0000 7f9165cbd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f916009dc60 0x7f91601f9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:33.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.931+0000 7f9165cbd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91600cd570 0x7f91600c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:33.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.931+0000 7f9165cbd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f916006f5d0 con 0x7f91600cd570 2024-09-17T04:33:33.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.931+0000 7f9165cbd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f916006f450 con 0x7f916006ee30 2024-09-17T04:33:33.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.931+0000 7f9165cbd640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f916006f750 con 0x7f916009dc60 2024-09-17T04:33:33.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.931+0000 7f915ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91600cd570 0x7f91600c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:33.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.931+0000 7f915ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91600cd570 0x7f91600c9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46118/0 (socket says 172.21.15.37:46118) 2024-09-17T04:33:33.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.931+0000 7f915ffff640 1 -- 172.21.15.37:0/1097343971 learned_addr learned my addr 172.21.15.37:0/1097343971 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:33.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.931+0000 7f915f7fe640 1 --2- 172.21.15.37:0/1097343971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f916006ee30 0x7f916009c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:33.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.931+0000 7f915effd640 1 --2- 172.21.15.37:0/1097343971 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f916009dc60 0x7f91601f9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:33.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.931+0000 7f915ffff640 1 -- 172.21.15.37:0/1097343971 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f916009dc60 msgr2=0x7f91601f9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:33.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.931+0000 7f915ffff640 1 --2- 172.21.15.37:0/1097343971 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f916009dc60 0x7f91601f9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:33.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.931+0000 7f915ffff640 1 -- 172.21.15.37:0/1097343971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f916006ee30 msgr2=0x7f916009c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:33.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.931+0000 7f915ffff640 1 --2- 172.21.15.37:0/1097343971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f916006ee30 0x7f916009c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:33.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.931+0000 7f915ffff640 1 -- 172.21.15.37:0/1097343971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f916006fa70 con 0x7f91600cd570 2024-09-17T04:33:33.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.931+0000 7f915ffff640 1 --2- 172.21.15.37:0/1097343971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91600cd570 0x7f91600c9550 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f915800c730 tx=0x7f915800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:33.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.931+0000 7f9147fff640 1 -- 172.21.15.37:0/1097343971 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9158003b70 con 0x7f91600cd570 2024-09-17T04:33:33.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.931+0000 7f9147fff640 1 -- 172.21.15.37:0/1097343971 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9158003d30 con 0x7f91600cd570 2024-09-17T04:33:33.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.931+0000 7f9165cbd640 1 -- 172.21.15.37:0/1097343971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f916006fd50 con 0x7f91600cd570 2024-09-17T04:33:33.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.931+0000 7f9147fff640 1 -- 172.21.15.37:0/1097343971 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91580052d0 con 0x7f91600cd570 2024-09-17T04:33:33.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.932+0000 7f9165cbd640 1 -- 172.21.15.37:0/1097343971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9160070310 con 0x7f91600cd570 2024-09-17T04:33:33.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.933+0000 7f9165cbd640 1 -- 172.21.15.37:0/1097343971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91601f5bc0 con 0x7f91600cd570 2024-09-17T04:33:33.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.933+0000 7f9147fff640 1 -- 172.21.15.37:0/1097343971 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9158005680 con 0x7f91600cd570 2024-09-17T04:33:33.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.934+0000 7f9147fff640 1 --2- 172.21.15.37:0/1097343971 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f914c078b90 0x7f914c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:33.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.934+0000 7f9147fff640 1 -- 172.21.15.37:0/1097343971 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f915809abd0 con 0x7f91600cd570 2024-09-17T04:33:33.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.934+0000 7f915f7fe640 1 --2- 172.21.15.37:0/1097343971 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f914c078b90 0x7f914c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:33.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.935+0000 7f915f7fe640 1 --2- 172.21.15.37:0/1097343971 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f914c078b90 0x7f914c07b0a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9150034000 tx=0x7f9150004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:33.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:33.938+0000 7f9147fff640 1 -- 172.21.15.37:0/1097343971 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f91580629b0 con 0x7f91600cd570 2024-09-17T04:33:34.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:34.161+0000 7f9165cbd640 1 -- 172.21.15.37:0/1097343971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f91600707e0 con 0x7f91600cd570 2024-09-17T04:33:34.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:34.161+0000 7f9147fff640 1 -- 172.21.15.37:0/1097343971 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f91580673e0 con 0x7f91600cd570 2024-09-17T04:33:34.161 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:33:34.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:34.164+0000 7f9165cbd640 1 -- 172.21.15.37:0/1097343971 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f914c078b90 msgr2=0x7f914c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:34.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:34.164+0000 7f9165cbd640 1 --2- 172.21.15.37:0/1097343971 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f914c078b90 0x7f914c07b0a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9150034000 tx=0x7f9150004730 comp rx=0 tx=0).stop 2024-09-17T04:33:34.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:34.164+0000 7f9165cbd640 1 -- 172.21.15.37:0/1097343971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91600cd570 msgr2=0x7f91600c9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:34.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:34.164+0000 7f9165cbd640 1 --2- 172.21.15.37:0/1097343971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91600cd570 0x7f91600c9550 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f915800c730 tx=0x7f915800cbf0 comp rx=0 tx=0).stop 2024-09-17T04:33:34.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:34.164+0000 7f9165cbd640 1 -- 172.21.15.37:0/1097343971 shutdown_connections 2024-09-17T04:33:34.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:34.164+0000 7f9165cbd640 1 --2- 172.21.15.37:0/1097343971 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f914c078b90 0x7f914c07b0a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:34.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:34.164+0000 7f9165cbd640 1 --2- 172.21.15.37:0/1097343971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91600cd570 0x7f91600c9550 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:34.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:34.164+0000 7f9165cbd640 1 --2- 172.21.15.37:0/1097343971 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f916009dc60 0x7f91601f9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:34.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:34.164+0000 7f9165cbd640 1 --2- 172.21.15.37:0/1097343971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f916006ee30 0x7f916009c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:34.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:34.164+0000 7f9165cbd640 1 -- 172.21.15.37:0/1097343971 >> 172.21.15.37:0/1097343971 conn(0x7f91600b1ca0 msgr2=0x7f91600b42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:34.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:34.164+0000 7f9165cbd640 1 -- 172.21.15.37:0/1097343971 shutdown_connections 2024-09-17T04:33:34.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:34.164+0000 7f9165cbd640 1 -- 172.21.15.37:0/1097343971 wait complete. 2024-09-17T04:33:34.175 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T04:33:34.175 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:33:34.175 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-09-17T04:33:34.175 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-17T04:33:34.175 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-17T04:33:34.175 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T04:33:34.175 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T04:33:34.175 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:33:34.175 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-17T04:33:34.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:34.176 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-09-17T04:33:34.192 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:33:34.192 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:33:34.192 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:33:34.192 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:34.192 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-09-17T04:33:34.192 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:34.192 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:33:34.192 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:34.192 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-09-17T04:33:34.193 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:33:34.193 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:34.193 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-09-17T04:33:34.193 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:33:34.193 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:34.193 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-09-17T04:33:34.193 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-17T04:33:34.193 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42738sec preferred_lft 42738sec 2024-09-17T04:33:34.193 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-09-17T04:33:34.193 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:34.193 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:33:34.193 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:34.193 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-09-17T04:33:34.193 INFO:teuthology.orchestra.run.smithi037.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:33:34.194 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether d6:3e:49:21:b4:0b brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:34.194 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:33:34.194 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:34.194 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::8fb:ceff:fe9f:ed7d/64 scope link 2024-09-17T04:33:34.194 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:34.194 INFO:teuthology.orchestra.run.smithi037.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:33:34.194 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether d6:3e:49:21:b4:0b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:33:34.194 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::d43e:49ff:fe21:b40b/64 scope link tentative 2024-09-17T04:33:34.194 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:34.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:34.195 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:33:34.257 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-09-17T04:33:34.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:34.259 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-09-17T04:33:34.315 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-09-17T04:33:34.316 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:33:34.316 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-17T04:33:34.316 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-17T04:33:34.316 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:33:34.316 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:33:34.316 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-17T04:33:34.317 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T04:33:34.317 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T04:33:34.317 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T04:33:34.317 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-17T04:33:34.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:33:34.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:33:34.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:34.445 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-09-17T04:33:34.461 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T04:33:34.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:34.462 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list-id 2024-09-17T04:33:34.518 INFO:teuthology.orchestra.run.smithi037.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-17T04:33:34.518 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:33:34.519 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-17T04:33:34.519 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:33:34.519 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-17T04:33:34.519 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-17T04:33:34.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:33:34.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:33:34.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:34.637 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-17T04:33:34.663 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:33:34.663 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:33:34.663 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:33:34.663 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:34.663 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-09-17T04:33:34.663 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:34.663 INFO:teuthology.orchestra.run.smithi037.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:33:34.664 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 6e:a7:ac:fd:9b:4f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T04:33:34.664 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T04:33:34.664 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:34.664 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::6ca7:acff:fefd:9b4f/64 scope link 2024-09-17T04:33:34.664 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:34.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:34.665 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-17T04:33:34.731 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:33:34.731 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:33:34.731 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:33:34.731 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:34.732 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-09-17T04:33:34.732 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:34.732 INFO:teuthology.orchestra.run.smithi037.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:33:34.732 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 6e:a7:ac:fd:9b:4f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T04:33:34.732 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T04:33:34.732 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:34.732 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::6ca7:acff:fefd:9b4f/64 scope link 2024-09-17T04:33:34.732 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:34.733 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-17T04:33:34.734 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:33:34.734 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-17T04:33:34.734 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-17T04:33:34.734 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-17T04:33:34.734 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-17T04:33:34.734 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-17T04:33:34.734 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-17T04:33:34.734 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-17T04:33:34.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:33:34.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:33:34.890 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:33:34.890 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-17T04:33:34.890 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.1 up 2024-09-17T04:33:34.890 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set dev brx.1 master ceph-brx 2024-09-17T04:33:34.891 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-17T04:33:34.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:33:34.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:33:34.969 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-09-17T04:33:34.969 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:33:34.970 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-17T04:33:34.985 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-17T04:33:34.985 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:33:34.985 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-17T04:33:35.041 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-09-17T04:33:35.108 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T04:33:35.165 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-09-17T04:33:35.165 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-09-17T04:33:35.165 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-09-17T04:33:35.165 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-09-17T04:33:35.165 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-09-17T04:33:35.165 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-09-17T04:33:35.165 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-09-17T04:33:35.165 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-09-17T04:33:35.165 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-09-17T04:33:35.165 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-09-17T04:33:35.165 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-09-17T04:33:35.165 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/selinux 2024-09-17T04:33:35.166 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T04:33:35.166 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-09-17T04:33:35.166 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-09-17T04:33:35.166 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-09-17T04:33:35.166 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-09-17T04:33:35.166 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-09-17T04:33:35.166 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-09-17T04:33:35.166 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T04:33:35.166 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T04:33:35.166 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T04:33:35.166 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T04:33:35.166 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-09-17T04:33:35.167 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-09-17T04:33:35.167 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T04:33:35.167 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T04:33:35.167 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T04:33:35.167 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T04:33:35.167 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-09-17T04:33:35.167 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:33:35.167 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:33:35.167 INFO:teuthology.orchestra.run.smithi037.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-17T04:33:35.167 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:33:35.167 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:33:35.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:35.168 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-09-17T04:33:35.221 INFO:teuthology.orchestra.run.smithi037.stdout:42 2024-09-17T04:33:35.221 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-09-17T04:33:35.221 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-09-17T04:33:35.264 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-09-17T04:33:35.289 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T04:33:35.348 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-09-17T04:33:35.348 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-09-17T04:33:35.348 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-09-17T04:33:35.348 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-09-17T04:33:35.349 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-09-17T04:33:35.349 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-09-17T04:33:35.349 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-09-17T04:33:35.349 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-09-17T04:33:35.349 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-09-17T04:33:35.349 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-09-17T04:33:35.349 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-09-17T04:33:35.349 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/selinux 2024-09-17T04:33:35.349 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T04:33:35.350 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-09-17T04:33:35.350 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-09-17T04:33:35.350 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-09-17T04:33:35.350 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-09-17T04:33:35.350 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-09-17T04:33:35.350 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-09-17T04:33:35.350 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T04:33:35.350 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T04:33:35.350 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T04:33:35.350 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T04:33:35.350 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-09-17T04:33:35.350 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-09-17T04:33:35.350 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T04:33:35.350 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T04:33:35.351 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T04:33:35.351 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T04:33:35.351 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-09-17T04:33:35.351 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:33:35.351 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:33:35.351 INFO:teuthology.orchestra.run.smithi037.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-17T04:33:35.351 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:33:35.351 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:33:35.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:35.351 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-09-17T04:33:35.359 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi037.stderr:2024-09-17T04:33:35.359+0000 7feaf172a480 -1 init, newargv = 0x7feae0004580 newargc=15 2024-09-17T04:33:35.359 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi037.stderr:ceph-fuse[45821]: starting ceph client 2024-09-17T04:33:35.364 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi037.stderr:ceph-fuse[45821]: starting fuse 2024-09-17T04:33:35.375 INFO:teuthology.orchestra.run.smithi037.stdout:42 2024-09-17T04:33:35.375 INFO:teuthology.orchestra.run.smithi037.stdout:44 2024-09-17T04:33:35.375 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 44] 2024-09-17T04:33:35.375 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-09-17T04:33:35.375 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-09-17T04:33:35.376 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-09-17T04:33:35.376 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-17T04:33:35.376 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-09-17T04:33:35.376 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-17T04:33:35.376 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-09-17T04:33:35.376 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-09-17T04:33:35.376 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-09-17T04:33:35.376 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-09-17T04:33:35.376 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-17T04:33:35.376 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-09-17T04:33:35.376 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-09-17T04:33:35.376 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-09-17T04:33:35.376 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-09-17T04:33:35.376 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-17T04:33:35.377 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-09-17T04:33:35.377 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T04:33:35.377 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T04:33:35.377 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T04:33:35.377 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-09-17T04:33:35.377 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-09-17T04:33:35.377 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-09-17T04:33:35.377 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T04:33:35.377 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-17T04:33:35.377 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.2")) 2024-09-17T04:33:35.377 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-17T04:33:35.495 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.2.45821.asok 2024-09-17T04:33:35.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:33:35.506 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.45821.asok 2024-09-17T04:33:35.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:35.506 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.45821.asok status 2024-09-17T04:33:35.652 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T04:33:35.653 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-09-17T04:33:35.653 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T04:33:35.653 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T04:33:35.653 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "2", 2024-09-17T04:33:35.653 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-09-17T04:33:35.653 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-09-17T04:33:35.653 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "45821", 2024-09-17T04:33:35.653 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-09-17T04:33:35.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:35.653 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-09-17T04:33:35.653 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-09-17T04:33:35.654 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 4752, 2024-09-17T04:33:35.654 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-09-17T04:33:35.654 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-09-17T04:33:35.654 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-09-17T04:33:35.654 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 4752 2024-09-17T04:33:35.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:35.654 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-09-17T04:33:35.654 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-09-17T04:33:35.654 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.2:0", 2024-09-17T04:33:35.654 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 144242247 2024-09-17T04:33:35.654 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:35.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:35.654 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-09-17T04:33:35.654 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-09-17T04:33:35.655 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.2:0", 2024-09-17T04:33:35.655 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 144242247 2024-09-17T04:33:35.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:35.655 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.4752 192.168.0.2:0/144242247", 2024-09-17T04:33:35.655 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.2:0/144242247", 2024-09-17T04:33:35.655 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-09-17T04:33:35.655 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 8, 2024-09-17T04:33:35.655 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 22, 2024-09-17T04:33:35.655 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-09-17T04:33:35.655 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-09-17T04:33:35.655 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-09-17T04:33:35.655 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T04:33:35.661 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-17T04:33:35.661 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:33:35.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.781+0000 7ff176a0e640 1 -- 172.21.15.37:0/2521236352 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff1700b2560 msgr2=0x7ff1700d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:35.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.781+0000 7ff176a0e640 1 --2- 172.21.15.37:0/2521236352 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff1700b2560 0x7ff1700d1e20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff160005fd0 tx=0x7ff16002a540 comp rx=0 tx=0).stop 2024-09-17T04:33:35.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.781+0000 7ff176a0e640 1 -- 172.21.15.37:0/2521236352 shutdown_connections 2024-09-17T04:33:35.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.781+0000 7ff176a0e640 1 --2- 172.21.15.37:0/2521236352 >> v2:172.21.15.92:3300/0 conn(0x7ff1700ce210 0x7ff1700cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:35.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.781+0000 7ff176a0e640 1 --2- 172.21.15.37:0/2521236352 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff1700b2560 0x7ff1700d1e20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:35.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.781+0000 7ff176a0e640 1 -- 172.21.15.37:0/2521236352 >> 172.21.15.37:0/2521236352 conn(0x7ff1700d1300 msgr2=0x7ff1700d1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:35.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.781+0000 7ff176a0e640 1 -- 172.21.15.37:0/2521236352 shutdown_connections 2024-09-17T04:33:35.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.781+0000 7ff176a0e640 1 -- 172.21.15.37:0/2521236352 wait complete. 2024-09-17T04:33:35.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.783+0000 7ff176a0e640 1 Processor -- start 2024-09-17T04:33:35.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.783+0000 7ff176a0e640 1 -- start start 2024-09-17T04:33:35.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.784+0000 7ff176a0e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff1700b2560 0x7ff170156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:35.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.784+0000 7ff176a0e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1700ce210 0x7ff1701e16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:35.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.784+0000 7ff176a0e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1700cfd00 0x7ff170159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:35.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.784+0000 7ff176a0e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff1700b1080 con 0x7ff1700ce210 2024-09-17T04:33:35.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.784+0000 7ff176a0e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff1700b60a0 con 0x7ff1700cfd00 2024-09-17T04:33:35.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.784+0000 7ff176a0e640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7ff1700b1200 con 0x7ff1700b2560 2024-09-17T04:33:35.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.784+0000 7ff16f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1700ce210 0x7ff1701e16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:35.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.784+0000 7ff16f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1700ce210 0x7ff1701e16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52918/0 (socket says 172.21.15.37:52918) 2024-09-17T04:33:35.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.784+0000 7ff16f7fe640 1 -- 172.21.15.37:0/1655992649 learned_addr learned my addr 172.21.15.37:0/1655992649 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:35.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.784+0000 7ff16ffff640 1 --2- 172.21.15.37:0/1655992649 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff1700b2560 0x7ff170156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:35.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.784+0000 7ff174fab640 1 --2- 172.21.15.37:0/1655992649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1700cfd00 0x7ff170159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:35.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.785+0000 7ff16f7fe640 1 -- 172.21.15.37:0/1655992649 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff1700b2560 msgr2=0x7ff170156980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:35.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.785+0000 7ff16f7fe640 1 --2- 172.21.15.37:0/1655992649 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff1700b2560 0x7ff170156980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:35.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.785+0000 7ff16f7fe640 1 -- 172.21.15.37:0/1655992649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1700cfd00 msgr2=0x7ff170159200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:35.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.785+0000 7ff16f7fe640 1 --2- 172.21.15.37:0/1655992649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1700cfd00 0x7ff170159200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:35.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.785+0000 7ff16f7fe640 1 -- 172.21.15.37:0/1655992649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1701e22d0 con 0x7ff1700ce210 2024-09-17T04:33:35.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.785+0000 7ff16ffff640 1 --2- 172.21.15.37:0/1655992649 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff1700b2560 0x7ff170156980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:33:35.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.785+0000 7ff174fab640 1 --2- 172.21.15.37:0/1655992649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1700cfd00 0x7ff170159200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:33:35.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.785+0000 7ff16f7fe640 1 --2- 172.21.15.37:0/1655992649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1700ce210 0x7ff1701e16f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff16400ea20 tx=0x7ff16400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:35.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.785+0000 7ff16cff9640 1 -- 172.21.15.37:0/1655992649 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16400ce40 con 0x7ff1700ce210 2024-09-17T04:33:35.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.785+0000 7ff16cff9640 1 -- 172.21.15.37:0/1655992649 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff164004020 con 0x7ff1700ce210 2024-09-17T04:33:35.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.785+0000 7ff176a0e640 1 -- 172.21.15.37:0/1655992649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1701e25b0 con 0x7ff1700ce210 2024-09-17T04:33:35.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.785+0000 7ff16cff9640 1 -- 172.21.15.37:0/1655992649 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff164004470 con 0x7ff1700ce210 2024-09-17T04:33:35.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.785+0000 7ff176a0e640 1 -- 172.21.15.37:0/1655992649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1701e2b70 con 0x7ff1700ce210 2024-09-17T04:33:35.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.786+0000 7ff176a0e640 1 -- 172.21.15.37:0/1655992649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1700b1720 con 0x7ff1700ce210 2024-09-17T04:33:35.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.787+0000 7ff16cff9640 1 -- 172.21.15.37:0/1655992649 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff1640035f0 con 0x7ff1700ce210 2024-09-17T04:33:35.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.787+0000 7ff16cff9640 1 --2- 172.21.15.37:0/1655992649 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff15c078bc0 0x7ff15c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:35.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.788+0000 7ff16cff9640 1 -- 172.21.15.37:0/1655992649 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff16409b790 con 0x7ff1700ce210 2024-09-17T04:33:35.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.788+0000 7ff16ffff640 1 --2- 172.21.15.37:0/1655992649 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff15c078bc0 0x7ff15c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:35.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.788+0000 7ff16ffff640 1 --2- 172.21.15.37:0/1655992649 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff15c078bc0 0x7ff15c07b0d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff160033000 tx=0x7ff160004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:35.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:35.792+0000 7ff16cff9640 1 -- 172.21.15.37:0/1655992649 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff1640634f0 con 0x7ff1700ce210 2024-09-17T04:33:36.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:36.018+0000 7ff176a0e640 1 -- 172.21.15.37:0/1655992649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff1701e2060 con 0x7ff1700ce210 2024-09-17T04:33:36.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:36.019+0000 7ff16cff9640 1 -- 172.21.15.37:0/1655992649 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7ff164067f20 con 0x7ff1700ce210 2024-09-17T04:33:36.019 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:33:36.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:36.021+0000 7ff176a0e640 1 -- 172.21.15.37:0/1655992649 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff15c078bc0 msgr2=0x7ff15c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:36.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:36.021+0000 7ff176a0e640 1 --2- 172.21.15.37:0/1655992649 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff15c078bc0 0x7ff15c07b0d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff160033000 tx=0x7ff160004730 comp rx=0 tx=0).stop 2024-09-17T04:33:36.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:36.021+0000 7ff176a0e640 1 -- 172.21.15.37:0/1655992649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1700ce210 msgr2=0x7ff1701e16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:36.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:36.021+0000 7ff176a0e640 1 --2- 172.21.15.37:0/1655992649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1700ce210 0x7ff1701e16f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff16400ea20 tx=0x7ff16400eee0 comp rx=0 tx=0).stop 2024-09-17T04:33:36.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:36.021+0000 7ff176a0e640 1 -- 172.21.15.37:0/1655992649 shutdown_connections 2024-09-17T04:33:36.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:36.021+0000 7ff176a0e640 1 --2- 172.21.15.37:0/1655992649 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff15c078bc0 0x7ff15c07b0d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:36.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:36.021+0000 7ff176a0e640 1 --2- 172.21.15.37:0/1655992649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1700cfd00 0x7ff170159200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:36.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:36.021+0000 7ff176a0e640 1 --2- 172.21.15.37:0/1655992649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1700ce210 0x7ff1701e16f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:36.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:36.021+0000 7ff176a0e640 1 --2- 172.21.15.37:0/1655992649 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff1700b2560 0x7ff170156980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:36.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:36.021+0000 7ff176a0e640 1 -- 172.21.15.37:0/1655992649 >> 172.21.15.37:0/1655992649 conn(0x7ff1700d1300 msgr2=0x7ff1700b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:36.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:36.021+0000 7ff176a0e640 1 -- 172.21.15.37:0/1655992649 shutdown_connections 2024-09-17T04:33:36.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:36.021+0000 7ff176a0e640 1 -- 172.21.15.37:0/1655992649 wait complete. 2024-09-17T04:33:36.032 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T04:33:36.032 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:33:36.032 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2024-09-17T04:33:36.032 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T04:33:36.032 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T04:33:36.032 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T04:33:36.032 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T04:33:36.032 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:33:36.033 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:33:36.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:36.033 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2024-09-17T04:33:36.053 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:33:36.053 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:33:36.053 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:33:36.053 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:36.053 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2024-09-17T04:33:36.053 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:36.053 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:33:36.053 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:36.053 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2024-09-17T04:33:36.053 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:33:36.054 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:36.054 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2024-09-17T04:33:36.054 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:33:36.054 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:36.054 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:33:36.054 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:36.054 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:33:36.054 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42779sec preferred_lft 42779sec 2024-09-17T04:33:36.054 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link noprefixroute 2024-09-17T04:33:36.054 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:36.054 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-17T04:33:36.055 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:33:36.055 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-09-17T04:33:36.055 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add name ceph-brx type bridge 2024-09-17T04:33:36.055 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr flush dev ceph-brx 2024-09-17T04:33:36.055 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set ceph-brx up 2024-09-17T04:33:36.055 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-17T04:33:36.055 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-09-17T04:33:36.145 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:33:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:33:36.334 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:33:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:33:36.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:36.342 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:33:36.365 INFO:teuthology.orchestra.run.smithi092.stdout:1 2024-09-17T04:33:36.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:36.367 DEBUG:teuthology.orchestra.run.smithi092:> route 2024-09-17T04:33:36.430 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2024-09-17T04:33:36.430 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:33:36.430 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:33:36.430 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:33:36.430 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:33:36.430 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:33:36.430 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-09-17T04:33:36.430 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:33:36.431 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:33:36.431 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:33:36.431 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-09-17T04:33:36.517 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:33:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:33:36.737 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:33:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:33:36.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:36.744 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2024-09-17T04:33:36.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:36.760 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list-id 2024-09-17T04:33:36.815 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:33:36.815 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-09-17T04:33:36.815 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T04:33:36.816 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-17T04:33:36.816 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-09-17T04:33:36.903 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:33:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:33:36.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:33:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:33:36.934 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-17T04:33:36.934 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:33:36.934 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-09-17T04:33:36.934 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-17T04:33:36.934 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-17T04:33:36.934 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-17T04:33:36.934 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-17T04:33:36.934 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-17T04:33:36.935 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-09-17T04:33:36.983 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:33:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:33:37.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:33:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:33:37.084 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:33:37.084 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-09-17T04:33:37.084 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2024-09-17T04:33:37.084 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set dev brx.0 master ceph-brx 2024-09-17T04:33:37.085 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-09-17T04:33:37.132 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:33:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:33:37.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:33:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:33:37.162 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-17T04:33:37.162 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:33:37.162 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T04:33:37.190 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T04:33:37.190 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:33:37.191 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T04:33:37.246 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2024-09-17T04:33:37.313 DEBUG:teuthology.orchestra.run.smithi092:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T04:33:37.370 INFO:teuthology.orchestra.run.smithi092.stdout:/proc 2024-09-17T04:33:37.370 INFO:teuthology.orchestra.run.smithi092.stdout:/sys 2024-09-17T04:33:37.370 INFO:teuthology.orchestra.run.smithi092.stdout:/dev 2024-09-17T04:33:37.370 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/security 2024-09-17T04:33:37.370 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/shm 2024-09-17T04:33:37.370 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/pts 2024-09-17T04:33:37.370 INFO:teuthology.orchestra.run.smithi092.stdout:/run 2024-09-17T04:33:37.370 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/cgroup 2024-09-17T04:33:37.370 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/pstore 2024-09-17T04:33:37.371 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/bpf 2024-09-17T04:33:37.371 INFO:teuthology.orchestra.run.smithi092.stdout:/ 2024-09-17T04:33:37.371 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/selinux 2024-09-17T04:33:37.371 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T04:33:37.371 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/mqueue 2024-09-17T04:33:37.371 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/hugepages 2024-09-17T04:33:37.371 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/debug 2024-09-17T04:33:37.371 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/tracing 2024-09-17T04:33:37.371 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/fuse/connections 2024-09-17T04:33:37.371 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/config 2024-09-17T04:33:37.372 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T04:33:37.372 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T04:33:37.372 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T04:33:37.372 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T04:33:37.372 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/1000 2024-09-17T04:33:37.372 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph 2024-09-17T04:33:37.372 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-4 2024-09-17T04:33:37.372 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-5 2024-09-17T04:33:37.372 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-6 2024-09-17T04:33:37.372 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-7 2024-09-17T04:33:37.373 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns 2024-09-17T04:33:37.373 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T04:33:37.373 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T04:33:37.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:37.373 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2024-09-17T04:33:37.425 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T04:33:37.425 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-17T04:33:37.468 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2024-09-17T04:33:37.494 DEBUG:teuthology.orchestra.run.smithi092:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T04:33:37.550 INFO:teuthology.orchestra.run.smithi092.stdout:/proc 2024-09-17T04:33:37.550 INFO:teuthology.orchestra.run.smithi092.stdout:/sys 2024-09-17T04:33:37.551 INFO:teuthology.orchestra.run.smithi092.stdout:/dev 2024-09-17T04:33:37.551 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/security 2024-09-17T04:33:37.551 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/shm 2024-09-17T04:33:37.551 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/pts 2024-09-17T04:33:37.551 INFO:teuthology.orchestra.run.smithi092.stdout:/run 2024-09-17T04:33:37.551 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/cgroup 2024-09-17T04:33:37.551 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/pstore 2024-09-17T04:33:37.551 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/bpf 2024-09-17T04:33:37.551 INFO:teuthology.orchestra.run.smithi092.stdout:/ 2024-09-17T04:33:37.551 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/selinux 2024-09-17T04:33:37.551 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T04:33:37.552 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/mqueue 2024-09-17T04:33:37.552 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/hugepages 2024-09-17T04:33:37.552 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/debug 2024-09-17T04:33:37.552 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/tracing 2024-09-17T04:33:37.552 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/fuse/connections 2024-09-17T04:33:37.552 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/config 2024-09-17T04:33:37.552 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T04:33:37.552 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T04:33:37.552 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T04:33:37.552 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T04:33:37.553 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/1000 2024-09-17T04:33:37.553 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph 2024-09-17T04:33:37.553 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-4 2024-09-17T04:33:37.553 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-5 2024-09-17T04:33:37.553 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-6 2024-09-17T04:33:37.553 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-7 2024-09-17T04:33:37.553 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns 2024-09-17T04:33:37.553 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T04:33:37.553 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T04:33:37.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:37.554 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2024-09-17T04:33:37.570 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi092.stderr:2024-09-17T04:33:37.570+0000 7efd34527480 -1 init, newargv = 0x7efd18003020 newargc=15 2024-09-17T04:33:37.570 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi092.stderr:ceph-fuse[42417]: starting ceph client 2024-09-17T04:33:37.575 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi092.stderr:ceph-fuse[42417]: starting fuse 2024-09-17T04:33:37.585 INFO:teuthology.orchestra.run.smithi092.stdout:41 2024-09-17T04:33:37.586 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-17T04:33:37.586 DEBUG:teuthology.orchestra.run.smithi092:> sudo stdin-killer -- python3 -c ' 2024-09-17T04:33:37.586 DEBUG:teuthology.orchestra.run.smithi092:> import glob 2024-09-17T04:33:37.586 DEBUG:teuthology.orchestra.run.smithi092:> import re 2024-09-17T04:33:37.586 DEBUG:teuthology.orchestra.run.smithi092:> import os 2024-09-17T04:33:37.586 DEBUG:teuthology.orchestra.run.smithi092:> import subprocess 2024-09-17T04:33:37.586 DEBUG:teuthology.orchestra.run.smithi092:> 2024-09-17T04:33:37.586 DEBUG:teuthology.orchestra.run.smithi092:> def _find_admin_socket(client_name): 2024-09-17T04:33:37.586 DEBUG:teuthology.orchestra.run.smithi092:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-17T04:33:37.586 DEBUG:teuthology.orchestra.run.smithi092:> files = glob.glob(asok_path) 2024-09-17T04:33:37.587 DEBUG:teuthology.orchestra.run.smithi092:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-17T04:33:37.587 DEBUG:teuthology.orchestra.run.smithi092:> 2024-09-17T04:33:37.587 DEBUG:teuthology.orchestra.run.smithi092:> # Given a non-glob path, it better be there 2024-09-17T04:33:37.587 DEBUG:teuthology.orchestra.run.smithi092:> if "*" not in asok_path: 2024-09-17T04:33:37.587 DEBUG:teuthology.orchestra.run.smithi092:> assert(len(files) == 1) 2024-09-17T04:33:37.587 DEBUG:teuthology.orchestra.run.smithi092:> return files[0] 2024-09-17T04:33:37.587 DEBUG:teuthology.orchestra.run.smithi092:> 2024-09-17T04:33:37.587 DEBUG:teuthology.orchestra.run.smithi092:> for f in files: 2024-09-17T04:33:37.587 DEBUG:teuthology.orchestra.run.smithi092:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T04:33:37.587 DEBUG:teuthology.orchestra.run.smithi092:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T04:33:37.587 DEBUG:teuthology.orchestra.run.smithi092:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T04:33:37.587 DEBUG:teuthology.orchestra.run.smithi092:> contents = proc_f.read() 2024-09-17T04:33:37.587 DEBUG:teuthology.orchestra.run.smithi092:> if mountpoint in contents: 2024-09-17T04:33:37.587 DEBUG:teuthology.orchestra.run.smithi092:> return f 2024-09-17T04:33:37.588 DEBUG:teuthology.orchestra.run.smithi092:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T04:33:37.588 DEBUG:teuthology.orchestra.run.smithi092:> 2024-09-17T04:33:37.588 DEBUG:teuthology.orchestra.run.smithi092:> print(_find_admin_socket("client.1")) 2024-09-17T04:33:37.588 DEBUG:teuthology.orchestra.run.smithi092:> ' 2024-09-17T04:33:37.702 INFO:teuthology.orchestra.run.smithi092.stdout:/var/run/ceph/ceph-client.1.42417.asok 2024-09-17T04:33:37.708 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:33:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:33:37.713 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.42417.asok 2024-09-17T04:33:37.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:37.713 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.42417.asok status 2024-09-17T04:33:37.857 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-17T04:33:37.857 INFO:teuthology.orchestra.run.smithi092.stdout: "metadata": { 2024-09-17T04:33:37.857 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T04:33:37.857 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T04:33:37.857 INFO:teuthology.orchestra.run.smithi092.stdout: "entity_id": "1", 2024-09-17T04:33:37.857 INFO:teuthology.orchestra.run.smithi092.stdout: "hostname": "smithi092", 2024-09-17T04:33:37.857 INFO:teuthology.orchestra.run.smithi092.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-17T04:33:37.857 INFO:teuthology.orchestra.run.smithi092.stdout: "pid": "42417", 2024-09-17T04:33:37.857 INFO:teuthology.orchestra.run.smithi092.stdout: "root": "/" 2024-09-17T04:33:37.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-17T04:33:37.858 INFO:teuthology.orchestra.run.smithi092.stdout: "dentry_count": 0, 2024-09-17T04:33:37.858 INFO:teuthology.orchestra.run.smithi092.stdout: "dentry_pinned_count": 0, 2024-09-17T04:33:37.858 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 4544, 2024-09-17T04:33:37.858 INFO:teuthology.orchestra.run.smithi092.stdout: "inst": { 2024-09-17T04:33:37.859 INFO:teuthology.orchestra.run.smithi092.stdout: "name": { 2024-09-17T04:33:37.859 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "client", 2024-09-17T04:33:37.859 INFO:teuthology.orchestra.run.smithi092.stdout: "num": 4544 2024-09-17T04:33:37.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-17T04:33:37.859 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": { 2024-09-17T04:33:37.859 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-17T04:33:37.859 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "192.168.0.1:0", 2024-09-17T04:33:37.859 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2681247147 2024-09-17T04:33:37.859 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-17T04:33:37.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-17T04:33:37.859 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": { 2024-09-17T04:33:37.859 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-17T04:33:37.859 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "192.168.0.1:0", 2024-09-17T04:33:37.860 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2681247147 2024-09-17T04:33:37.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-17T04:33:37.860 INFO:teuthology.orchestra.run.smithi092.stdout: "inst_str": "client.4544 192.168.0.1:0/2681247147", 2024-09-17T04:33:37.860 INFO:teuthology.orchestra.run.smithi092.stdout: "addr_str": "192.168.0.1:0/2681247147", 2024-09-17T04:33:37.860 INFO:teuthology.orchestra.run.smithi092.stdout: "inode_count": 1, 2024-09-17T04:33:37.860 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_epoch": 8, 2024-09-17T04:33:37.860 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_epoch": 22, 2024-09-17T04:33:37.860 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_epoch_barrier": 0, 2024-09-17T04:33:37.860 INFO:teuthology.orchestra.run.smithi092.stdout: "blocklisted": false, 2024-09-17T04:33:37.860 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs" 2024-09-17T04:33:37.860 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-17T04:33:37.865 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-17T04:33:37.866 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:33:37.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:37.998+0000 7f911deec640 1 -- 172.21.15.37:0/3440785918 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f911815e8e0 msgr2=0x7f91180cef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:37.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:37.998+0000 7f911deec640 1 --2- 172.21.15.37:0/3440785918 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f911815e8e0 0x7f91180cef40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9110009be0 tx=0x7f911002a920 comp rx=0 tx=0).stop 2024-09-17T04:33:37.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:37.998+0000 7f911deec640 1 -- 172.21.15.37:0/3440785918 shutdown_connections 2024-09-17T04:33:37.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:37.998+0000 7f911deec640 1 --2- 172.21.15.37:0/3440785918 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f911815e8e0 0x7f91180cef40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:37.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:37.998+0000 7f911deec640 1 -- 172.21.15.37:0/3440785918 >> 172.21.15.37:0/3440785918 conn(0x7f91180d44c0 msgr2=0x7f91180d48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:37.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:37.998+0000 7f911deec640 1 -- 172.21.15.37:0/3440785918 shutdown_connections 2024-09-17T04:33:37.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:37.998+0000 7f911deec640 1 -- 172.21.15.37:0/3440785918 wait complete. 2024-09-17T04:33:38.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.000+0000 7f911deec640 1 Processor -- start 2024-09-17T04:33:38.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.000+0000 7f911deec640 1 -- start start 2024-09-17T04:33:38.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.002+0000 7f911deec640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f911815e8e0 0x7f911815a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:38.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.002+0000 7f911deec640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f911815a730 0x7f91180cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:38.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.002+0000 7f911deec640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f911815ac80 0x7f91180ce300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:38.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.002+0000 7f911deec640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f91180b8400 con 0x7f911815ac80 2024-09-17T04:33:38.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.002+0000 7f911deec640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f91180b8280 con 0x7f911815e8e0 2024-09-17T04:33:38.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.002+0000 7f911deec640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f91180b8580 con 0x7f911815a730 2024-09-17T04:33:38.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.002+0000 7f9117fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f911815ac80 0x7f91180ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:38.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.002+0000 7f9117fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f911815ac80 0x7f91180ce300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52920/0 (socket says 172.21.15.37:52920) 2024-09-17T04:33:38.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.002+0000 7f91177fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f911815e8e0 0x7f911815a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:38.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.002+0000 7f9116ffd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f911815a730 0x7f91180cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:38.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.002+0000 7f9117fff640 1 -- 172.21.15.37:0/4048620704 learned_addr learned my addr 172.21.15.37:0/4048620704 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:38.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.002+0000 7f9117fff640 1 -- 172.21.15.37:0/4048620704 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f911815a730 msgr2=0x7f91180cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:38.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.002+0000 7f9117fff640 1 --2- 172.21.15.37:0/4048620704 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f911815a730 0x7f91180cddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:38.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.002+0000 7f9117fff640 1 -- 172.21.15.37:0/4048620704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f911815e8e0 msgr2=0x7f911815a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:38.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.002+0000 7f9117fff640 1 --2- 172.21.15.37:0/4048620704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f911815e8e0 0x7f911815a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:38.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.002+0000 7f9117fff640 1 -- 172.21.15.37:0/4048620704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f911815a540 con 0x7f911815ac80 2024-09-17T04:33:38.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.002+0000 7f91177fe640 1 --2- 172.21.15.37:0/4048620704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f911815e8e0 0x7f911815a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:33:38.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.002+0000 7f9117fff640 1 --2- 172.21.15.37:0/4048620704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f911815ac80 0x7f91180ce300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9110009d00 tx=0x7f9110003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:38.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.003+0000 7f90fffff640 1 -- 172.21.15.37:0/4048620704 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110030c50 con 0x7f911815ac80 2024-09-17T04:33:38.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.003+0000 7f90fffff640 1 -- 172.21.15.37:0/4048620704 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f911003e070 con 0x7f911815ac80 2024-09-17T04:33:38.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.003+0000 7f90fffff640 1 -- 172.21.15.37:0/4048620704 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110032330 con 0x7f911815ac80 2024-09-17T04:33:38.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.003+0000 7f911deec640 1 -- 172.21.15.37:0/4048620704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91180cb500 con 0x7f911815ac80 2024-09-17T04:33:38.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.003+0000 7f911deec640 1 -- 172.21.15.37:0/4048620704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91181ea950 con 0x7f911815ac80 2024-09-17T04:33:38.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.004+0000 7f911deec640 1 -- 172.21.15.37:0/4048620704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91180d3fe0 con 0x7f911815ac80 2024-09-17T04:33:38.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.005+0000 7f90fffff640 1 -- 172.21.15.37:0/4048620704 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f91100324d0 con 0x7f911815ac80 2024-09-17T04:33:38.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.005+0000 7f90fffff640 1 --2- 172.21.15.37:0/4048620704 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9104078b90 0x7f910407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:38.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.005+0000 7f90fffff640 1 -- 172.21.15.37:0/4048620704 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f91100bc460 con 0x7f911815ac80 2024-09-17T04:33:38.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.005+0000 7f91177fe640 1 --2- 172.21.15.37:0/4048620704 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9104078b90 0x7f910407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:38.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.006+0000 7f91177fe640 1 --2- 172.21.15.37:0/4048620704 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9104078b90 0x7f910407b0a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9108001120 tx=0x7f9108009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:38.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.010+0000 7f90fffff640 1 -- 172.21.15.37:0/4048620704 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f911002d070 con 0x7f911815ac80 2024-09-17T04:33:38.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.237+0000 7f911deec640 1 -- 172.21.15.37:0/4048620704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f91180d4d90 con 0x7f911815ac80 2024-09-17T04:33:38.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.237+0000 7f90fffff640 1 -- 172.21.15.37:0/4048620704 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f9110084250 con 0x7f911815ac80 2024-09-17T04:33:38.237 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:33:38.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.239+0000 7f911deec640 1 -- 172.21.15.37:0/4048620704 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9104078b90 msgr2=0x7f910407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:38.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.239+0000 7f911deec640 1 --2- 172.21.15.37:0/4048620704 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9104078b90 0x7f910407b0a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9108001120 tx=0x7f9108009040 comp rx=0 tx=0).stop 2024-09-17T04:33:38.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.239+0000 7f911deec640 1 -- 172.21.15.37:0/4048620704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f911815ac80 msgr2=0x7f91180ce300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:38.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.239+0000 7f911deec640 1 --2- 172.21.15.37:0/4048620704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f911815ac80 0x7f91180ce300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9110009d00 tx=0x7f9110003ba0 comp rx=0 tx=0).stop 2024-09-17T04:33:38.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.239+0000 7f911deec640 1 -- 172.21.15.37:0/4048620704 shutdown_connections 2024-09-17T04:33:38.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.239+0000 7f911deec640 1 --2- 172.21.15.37:0/4048620704 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9104078b90 0x7f910407b0a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:38.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.239+0000 7f911deec640 1 --2- 172.21.15.37:0/4048620704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f911815ac80 0x7f91180ce300 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:38.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.239+0000 7f911deec640 1 --2- 172.21.15.37:0/4048620704 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f911815a730 0x7f91180cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:38.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.239+0000 7f911deec640 1 --2- 172.21.15.37:0/4048620704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f911815e8e0 0x7f911815a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:38.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.239+0000 7f911deec640 1 -- 172.21.15.37:0/4048620704 >> 172.21.15.37:0/4048620704 conn(0x7f91180d44c0 msgr2=0x7f91180b5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:38.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.239+0000 7f911deec640 1 -- 172.21.15.37:0/4048620704 shutdown_connections 2024-09-17T04:33:38.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:38.239+0000 7f911deec640 1 -- 172.21.15.37:0/4048620704 wait complete. 2024-09-17T04:33:38.250 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T04:33:38.250 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:33:38.250 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2024-09-17T04:33:38.250 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-17T04:33:38.251 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-17T04:33:38.251 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T04:33:38.251 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T04:33:38.251 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:33:38.251 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-17T04:33:38.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:38.251 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2024-09-17T04:33:38.267 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:33:38.268 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:33:38.268 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:33:38.268 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:38.268 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2024-09-17T04:33:38.268 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:38.268 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:33:38.268 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:38.268 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2024-09-17T04:33:38.268 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:33:38.268 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:38.268 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2024-09-17T04:33:38.268 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:33:38.269 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:38.269 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:33:38.269 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:38.269 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T04:33:38.269 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42776sec preferred_lft 42776sec 2024-09-17T04:33:38.269 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link noprefixroute 2024-09-17T04:33:38.269 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:38.269 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:33:38.269 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether aa:e6:1f:c9:b2:04 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:33:38.269 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:33:38.269 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:38.269 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::70d8:1fff:fe3c:7cb1/64 scope link 2024-09-17T04:33:38.269 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:38.270 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:33:38.270 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether aa:e6:1f:c9:b2:04 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T04:33:38.270 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::a8e6:1fff:fec9:b204/64 scope link tentative 2024-09-17T04:33:38.270 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:38.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:38.270 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:33:38.332 INFO:teuthology.orchestra.run.smithi092.stdout:1 2024-09-17T04:33:38.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:38.333 DEBUG:teuthology.orchestra.run.smithi092:> route 2024-09-17T04:33:38.390 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2024-09-17T04:33:38.391 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:33:38.391 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T04:33:38.391 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T04:33:38.391 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:33:38.391 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:33:38.391 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-09-17T04:33:38.392 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T04:33:38.392 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T04:33:38.392 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T04:33:38.392 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-09-17T04:33:38.478 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:33:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:33:38.511 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:33:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:33:38.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:38.517 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2024-09-17T04:33:38.532 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T04:33:38.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:38.533 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list-id 2024-09-17T04:33:38.588 INFO:teuthology.orchestra.run.smithi092.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-17T04:33:38.588 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:33:38.588 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-09-17T04:33:38.588 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T04:33:38.588 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-17T04:33:38.588 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-09-17T04:33:38.675 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:33:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:33:38.698 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:33:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:33:38.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:38.704 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-17T04:33:38.728 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:33:38.728 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:33:38.728 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:33:38.729 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:38.729 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2024-09-17T04:33:38.729 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:38.729 INFO:teuthology.orchestra.run.smithi092.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:33:38.729 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 52:52:dd:4a:5e:83 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T04:33:38.729 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T04:33:38.729 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:38.729 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::5052:ddff:fe4a:5e83/64 scope link tentative 2024-09-17T04:33:38.729 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:38.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:38.730 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-17T04:33:38.795 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:33:38.795 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:33:38.795 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:33:38.795 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:38.795 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2024-09-17T04:33:38.795 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:38.795 INFO:teuthology.orchestra.run.smithi092.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:33:38.795 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 52:52:dd:4a:5e:83 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T04:33:38.796 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T04:33:38.796 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:38.796 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::5052:ddff:fe4a:5e83/64 scope link tentative 2024-09-17T04:33:38.796 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:33:38.797 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-17T04:33:38.797 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:33:38.797 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-09-17T04:33:38.797 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-17T04:33:38.797 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-17T04:33:38.797 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-17T04:33:38.797 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-17T04:33:38.797 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-17T04:33:38.797 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-09-17T04:33:38.884 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:33:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:33:38.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:33:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:33:38.948 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:33:38.948 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2024-09-17T04:33:38.948 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.1 up 2024-09-17T04:33:38.949 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set dev brx.1 master ceph-brx 2024-09-17T04:33:38.949 DEBUG:teuthology.orchestra.run.smithi092:> ') 2024-09-17T04:33:38.997 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:33:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:33:39.019 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:33:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:33:39.026 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-09-17T04:33:39.026 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:33:39.026 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-17T04:33:39.052 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-17T04:33:39.052 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:33:39.053 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-17T04:33:39.107 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2024-09-17T04:33:39.174 DEBUG:teuthology.orchestra.run.smithi092:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T04:33:39.230 INFO:teuthology.orchestra.run.smithi092.stdout:/proc 2024-09-17T04:33:39.230 INFO:teuthology.orchestra.run.smithi092.stdout:/sys 2024-09-17T04:33:39.230 INFO:teuthology.orchestra.run.smithi092.stdout:/dev 2024-09-17T04:33:39.230 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/security 2024-09-17T04:33:39.230 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/shm 2024-09-17T04:33:39.230 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/pts 2024-09-17T04:33:39.230 INFO:teuthology.orchestra.run.smithi092.stdout:/run 2024-09-17T04:33:39.230 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/cgroup 2024-09-17T04:33:39.230 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/pstore 2024-09-17T04:33:39.230 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/bpf 2024-09-17T04:33:39.231 INFO:teuthology.orchestra.run.smithi092.stdout:/ 2024-09-17T04:33:39.231 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/selinux 2024-09-17T04:33:39.231 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T04:33:39.231 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/mqueue 2024-09-17T04:33:39.231 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/hugepages 2024-09-17T04:33:39.231 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/debug 2024-09-17T04:33:39.231 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/tracing 2024-09-17T04:33:39.231 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/fuse/connections 2024-09-17T04:33:39.231 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/config 2024-09-17T04:33:39.231 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T04:33:39.231 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T04:33:39.232 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T04:33:39.232 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T04:33:39.232 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/1000 2024-09-17T04:33:39.232 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph 2024-09-17T04:33:39.232 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-4 2024-09-17T04:33:39.233 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-5 2024-09-17T04:33:39.233 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-6 2024-09-17T04:33:39.233 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-7 2024-09-17T04:33:39.233 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns 2024-09-17T04:33:39.233 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T04:33:39.233 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T04:33:39.234 INFO:teuthology.orchestra.run.smithi092.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-17T04:33:39.234 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T04:33:39.234 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T04:33:39.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:39.235 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2024-09-17T04:33:39.285 INFO:teuthology.orchestra.run.smithi092.stdout:41 2024-09-17T04:33:39.286 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2024-09-17T04:33:39.286 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-09-17T04:33:39.328 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2024-09-17T04:33:39.352 DEBUG:teuthology.orchestra.run.smithi092:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T04:33:39.410 INFO:teuthology.orchestra.run.smithi092.stdout:/proc 2024-09-17T04:33:39.410 INFO:teuthology.orchestra.run.smithi092.stdout:/sys 2024-09-17T04:33:39.410 INFO:teuthology.orchestra.run.smithi092.stdout:/dev 2024-09-17T04:33:39.410 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/security 2024-09-17T04:33:39.410 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/shm 2024-09-17T04:33:39.410 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/pts 2024-09-17T04:33:39.410 INFO:teuthology.orchestra.run.smithi092.stdout:/run 2024-09-17T04:33:39.411 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/cgroup 2024-09-17T04:33:39.411 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/pstore 2024-09-17T04:33:39.411 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/bpf 2024-09-17T04:33:39.411 INFO:teuthology.orchestra.run.smithi092.stdout:/ 2024-09-17T04:33:39.411 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/selinux 2024-09-17T04:33:39.411 INFO:teuthology.orchestra.run.smithi092.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T04:33:39.411 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/mqueue 2024-09-17T04:33:39.411 INFO:teuthology.orchestra.run.smithi092.stdout:/dev/hugepages 2024-09-17T04:33:39.411 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/debug 2024-09-17T04:33:39.411 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/tracing 2024-09-17T04:33:39.411 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/fs/fuse/connections 2024-09-17T04:33:39.411 INFO:teuthology.orchestra.run.smithi092.stdout:/sys/kernel/config 2024-09-17T04:33:39.411 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T04:33:39.412 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T04:33:39.412 INFO:teuthology.orchestra.run.smithi092.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T04:33:39.412 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T04:33:39.412 INFO:teuthology.orchestra.run.smithi092.stdout:/run/user/1000 2024-09-17T04:33:39.412 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph 2024-09-17T04:33:39.412 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-4 2024-09-17T04:33:39.412 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-5 2024-09-17T04:33:39.412 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-6 2024-09-17T04:33:39.412 INFO:teuthology.orchestra.run.smithi092.stdout:/var/lib/ceph/osd/ceph-7 2024-09-17T04:33:39.412 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns 2024-09-17T04:33:39.412 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T04:33:39.412 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T04:33:39.412 INFO:teuthology.orchestra.run.smithi092.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-17T04:33:39.412 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T04:33:39.413 INFO:teuthology.orchestra.run.smithi092.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T04:33:39.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:39.413 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2024-09-17T04:33:39.419 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi092.stderr:2024-09-17T04:33:39.419+0000 7f044a6f2480 -1 init, newargv = 0x7f0438001080 newargc=15 2024-09-17T04:33:39.419 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi092.stderr:ceph-fuse[42872]: starting ceph client 2024-09-17T04:33:39.425 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi092.stderr:ceph-fuse[42872]: starting fuse 2024-09-17T04:33:39.434 INFO:teuthology.orchestra.run.smithi092.stdout:41 2024-09-17T04:33:39.434 INFO:teuthology.orchestra.run.smithi092.stdout:44 2024-09-17T04:33:39.434 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 44] 2024-09-17T04:33:39.434 DEBUG:teuthology.orchestra.run.smithi092:> sudo stdin-killer -- python3 -c ' 2024-09-17T04:33:39.435 DEBUG:teuthology.orchestra.run.smithi092:> import glob 2024-09-17T04:33:39.435 DEBUG:teuthology.orchestra.run.smithi092:> import re 2024-09-17T04:33:39.435 DEBUG:teuthology.orchestra.run.smithi092:> import os 2024-09-17T04:33:39.435 DEBUG:teuthology.orchestra.run.smithi092:> import subprocess 2024-09-17T04:33:39.435 DEBUG:teuthology.orchestra.run.smithi092:> 2024-09-17T04:33:39.435 DEBUG:teuthology.orchestra.run.smithi092:> def _find_admin_socket(client_name): 2024-09-17T04:33:39.435 DEBUG:teuthology.orchestra.run.smithi092:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-09-17T04:33:39.435 DEBUG:teuthology.orchestra.run.smithi092:> files = glob.glob(asok_path) 2024-09-17T04:33:39.435 DEBUG:teuthology.orchestra.run.smithi092:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-09-17T04:33:39.435 DEBUG:teuthology.orchestra.run.smithi092:> 2024-09-17T04:33:39.435 DEBUG:teuthology.orchestra.run.smithi092:> # Given a non-glob path, it better be there 2024-09-17T04:33:39.435 DEBUG:teuthology.orchestra.run.smithi092:> if "*" not in asok_path: 2024-09-17T04:33:39.435 DEBUG:teuthology.orchestra.run.smithi092:> assert(len(files) == 1) 2024-09-17T04:33:39.435 DEBUG:teuthology.orchestra.run.smithi092:> return files[0] 2024-09-17T04:33:39.436 DEBUG:teuthology.orchestra.run.smithi092:> 2024-09-17T04:33:39.436 DEBUG:teuthology.orchestra.run.smithi092:> for f in files: 2024-09-17T04:33:39.436 DEBUG:teuthology.orchestra.run.smithi092:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T04:33:39.436 DEBUG:teuthology.orchestra.run.smithi092:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T04:33:39.436 DEBUG:teuthology.orchestra.run.smithi092:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T04:33:39.436 DEBUG:teuthology.orchestra.run.smithi092:> contents = proc_f.read() 2024-09-17T04:33:39.436 DEBUG:teuthology.orchestra.run.smithi092:> if mountpoint in contents: 2024-09-17T04:33:39.436 DEBUG:teuthology.orchestra.run.smithi092:> return f 2024-09-17T04:33:39.436 DEBUG:teuthology.orchestra.run.smithi092:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T04:33:39.436 DEBUG:teuthology.orchestra.run.smithi092:> 2024-09-17T04:33:39.436 DEBUG:teuthology.orchestra.run.smithi092:> print(_find_admin_socket("client.3")) 2024-09-17T04:33:39.437 DEBUG:teuthology.orchestra.run.smithi092:> ' 2024-09-17T04:33:39.552 INFO:teuthology.orchestra.run.smithi092.stdout:/var/run/ceph/ceph-client.3.42872.asok 2024-09-17T04:33:39.555 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-17T04:33:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:33:39.562 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.42872.asok 2024-09-17T04:33:39.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:39.562 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.42872.asok status 2024-09-17T04:33:39.703 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-17T04:33:39.899 INFO:teuthology.orchestra.run.smithi092.stdout: "metadata": { 2024-09-17T04:33:39.899 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T04:33:39.899 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T04:33:39.899 INFO:teuthology.orchestra.run.smithi092.stdout: "entity_id": "3", 2024-09-17T04:33:39.899 INFO:teuthology.orchestra.run.smithi092.stdout: "hostname": "smithi092", 2024-09-17T04:33:39.899 INFO:teuthology.orchestra.run.smithi092.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-09-17T04:33:39.900 INFO:teuthology.orchestra.run.smithi092.stdout: "pid": "42872", 2024-09-17T04:33:39.900 INFO:teuthology.orchestra.run.smithi092.stdout: "root": "/" 2024-09-17T04:33:39.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-17T04:33:39.900 INFO:teuthology.orchestra.run.smithi092.stdout: "dentry_count": 0, 2024-09-17T04:33:39.900 INFO:teuthology.orchestra.run.smithi092.stdout: "dentry_pinned_count": 0, 2024-09-17T04:33:39.900 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 4556, 2024-09-17T04:33:39.900 INFO:teuthology.orchestra.run.smithi092.stdout: "inst": { 2024-09-17T04:33:39.900 INFO:teuthology.orchestra.run.smithi092.stdout: "name": { 2024-09-17T04:33:39.900 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "client", 2024-09-17T04:33:39.900 INFO:teuthology.orchestra.run.smithi092.stdout: "num": 4556 2024-09-17T04:33:39.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-17T04:33:39.900 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": { 2024-09-17T04:33:39.900 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-17T04:33:39.901 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "192.168.0.2:0", 2024-09-17T04:33:39.901 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1540841940 2024-09-17T04:33:39.901 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-17T04:33:39.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-17T04:33:39.901 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": { 2024-09-17T04:33:39.901 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-17T04:33:39.901 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "192.168.0.2:0", 2024-09-17T04:33:39.901 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1540841940 2024-09-17T04:33:39.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-17T04:33:39.901 INFO:teuthology.orchestra.run.smithi092.stdout: "inst_str": "client.4556 192.168.0.2:0/1540841940", 2024-09-17T04:33:39.902 INFO:teuthology.orchestra.run.smithi092.stdout: "addr_str": "192.168.0.2:0/1540841940", 2024-09-17T04:33:39.902 INFO:teuthology.orchestra.run.smithi092.stdout: "inode_count": 1, 2024-09-17T04:33:39.902 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_epoch": 8, 2024-09-17T04:33:39.902 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_epoch": 22, 2024-09-17T04:33:39.902 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_epoch_barrier": 0, 2024-09-17T04:33:39.902 INFO:teuthology.orchestra.run.smithi092.stdout: "blocklisted": false, 2024-09-17T04:33:39.902 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs" 2024-09-17T04:33:39.902 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-17T04:33:39.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:39.904 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-09-17T04:33:39.904 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-17T04:33:39.924 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-09-17T04:33:39.925 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-17T04:33:39.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:39.925 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:33:39.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:39.997 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-09-17T04:33:39.997 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.2 2024-09-17T04:33:40.057 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-09-17T04:33:40.057 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-09-17T04:33:40.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:40.057 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-17T04:33:40.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:40.129 DEBUG:teuthology.orchestra.run.smithi092:> stat --file-system '--printf=%T 2024-09-17T04:33:40.129 DEBUG:teuthology.orchestra.run.smithi092:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-17T04:33:40.145 INFO:teuthology.orchestra.run.smithi092.stdout:fuseblk 2024-09-17T04:33:40.146 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-17T04:33:40.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:40.146 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T04:33:40.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:40.212 DEBUG:teuthology.orchestra.run.smithi092:> stat --file-system '--printf=%T 2024-09-17T04:33:40.212 DEBUG:teuthology.orchestra.run.smithi092:> ' -- /home/ubuntu/cephtest/mnt.3 2024-09-17T04:33:40.268 INFO:teuthology.orchestra.run.smithi092.stdout:fuseblk 2024-09-17T04:33:40.269 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-09-17T04:33:40.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:40.269 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-17T04:33:40.333 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-17T04:33:40.376 INFO:tasks.cephfs_test_runner:Starting test: test_rebuild_simple (tasks.cephfs.test_recovery_pool.TestRecoveryPool) 2024-09-17T04:33:40.377 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:33:40.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.509+0000 7f23fbf58640 1 -- 172.21.15.37:0/3965247082 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f23f415e7c0 msgr2=0x7f23f40d2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:40.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.509+0000 7f23fbf58640 1 --2- 172.21.15.37:0/3965247082 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f23f415e7c0 0x7f23f40d2280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009860 tx=0x7f23e402a410 comp rx=0 tx=0).stop 2024-09-17T04:33:40.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.509+0000 7f23fbf58640 1 -- 172.21.15.37:0/3965247082 shutdown_connections 2024-09-17T04:33:40.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.509+0000 7f23fbf58640 1 --2- 172.21.15.37:0/3965247082 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f23f415e7c0 0x7f23f40d2280 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:40.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.509+0000 7f23fbf58640 1 -- 172.21.15.37:0/3965247082 >> 172.21.15.37:0/3965247082 conn(0x7f23f4072a80 msgr2=0x7f23f4072e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:40.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.509+0000 7f23fbf58640 1 -- 172.21.15.37:0/3965247082 shutdown_connections 2024-09-17T04:33:40.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.509+0000 7f23fbf58640 1 -- 172.21.15.37:0/3965247082 wait complete. 2024-09-17T04:33:40.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.511+0000 7f23fbf58640 1 Processor -- start 2024-09-17T04:33:40.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.511+0000 7f23fbf58640 1 -- start start 2024-09-17T04:33:40.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.512+0000 7f23fbf58640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f40d27c0 0x7f23f40b6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:40.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.512+0000 7f23fbf58640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23f415e7c0 0x7f23f41539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:40.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.512+0000 7f23fbf58640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f23f415ff90 0x7f23f40b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:40.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.512+0000 7f23fbf58640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f23f40cf730 con 0x7f23f40d27c0 2024-09-17T04:33:40.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.512+0000 7f23fbf58640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f23f40cf5b0 con 0x7f23f415e7c0 2024-09-17T04:33:40.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.512+0000 7f23fbf58640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f23f40cf8b0 con 0x7f23f415ff90 2024-09-17T04:33:40.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.512+0000 7f23f9cf4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f40d27c0 0x7f23f40b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:40.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.512+0000 7f23f9cf4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f40d27c0 0x7f23f40b6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52924/0 (socket says 172.21.15.37:52924) 2024-09-17T04:33:40.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.512+0000 7f23f9cf4640 1 -- 172.21.15.37:0/4150404640 learned_addr learned my addr 172.21.15.37:0/4150404640 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:40.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.512+0000 7f23f94f3640 1 --2- 172.21.15.37:0/4150404640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23f415e7c0 0x7f23f41539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:40.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.512+0000 7f23fa4f5640 1 --2- 172.21.15.37:0/4150404640 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f23f415ff90 0x7f23f40b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:40.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.513+0000 7f23f9cf4640 1 -- 172.21.15.37:0/4150404640 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f23f415ff90 msgr2=0x7f23f40b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:40.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.513+0000 7f23f9cf4640 1 --2- 172.21.15.37:0/4150404640 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f23f415ff90 0x7f23f40b59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:40.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.513+0000 7f23f9cf4640 1 -- 172.21.15.37:0/4150404640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23f415e7c0 msgr2=0x7f23f41539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:40.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.513+0000 7f23f9cf4640 1 --2- 172.21.15.37:0/4150404640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23f415e7c0 0x7f23f41539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:40.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.513+0000 7f23f9cf4640 1 -- 172.21.15.37:0/4150404640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f40b6ea0 con 0x7f23f40d27c0 2024-09-17T04:33:40.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.513+0000 7f23f9cf4640 1 --2- 172.21.15.37:0/4150404640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f40d27c0 0x7f23f40b6960 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f23f000c460 tx=0x7f23f000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:40.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.513+0000 7f23ea7fc640 1 -- 172.21.15.37:0/4150404640 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f00052d0 con 0x7f23f40d27c0 2024-09-17T04:33:40.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.513+0000 7f23ea7fc640 1 -- 172.21.15.37:0/4150404640 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23f0005490 con 0x7f23f40d27c0 2024-09-17T04:33:40.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.513+0000 7f23ea7fc640 1 -- 172.21.15.37:0/4150404640 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f0012870 con 0x7f23f40d27c0 2024-09-17T04:33:40.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.513+0000 7f23fbf58640 1 -- 172.21.15.37:0/4150404640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f4153ee0 con 0x7f23f40d27c0 2024-09-17T04:33:40.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.513+0000 7f23fbf58640 1 -- 172.21.15.37:0/4150404640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f40ae630 con 0x7f23f40d27c0 2024-09-17T04:33:40.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.514+0000 7f23fbf58640 1 -- 172.21.15.37:0/4150404640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23f40b7070 con 0x7f23f40d27c0 2024-09-17T04:33:40.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.515+0000 7f23ea7fc640 1 -- 172.21.15.37:0/4150404640 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f23f0012a10 con 0x7f23f40d27c0 2024-09-17T04:33:40.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.516+0000 7f23ea7fc640 1 --2- 172.21.15.37:0/4150404640 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f23d8078b90 0x7f23d807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:40.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.516+0000 7f23ea7fc640 1 -- 172.21.15.37:0/4150404640 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f23f009a8a0 con 0x7f23f40d27c0 2024-09-17T04:33:40.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.516+0000 7f23f94f3640 1 --2- 172.21.15.37:0/4150404640 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f23d8078b90 0x7f23d807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:40.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.517+0000 7f23f94f3640 1 --2- 172.21.15.37:0/4150404640 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f23d8078b90 0x7f23d807b0a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f23e4034000 tx=0x7f23e4004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:40.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.520+0000 7f23ea7fc640 1 -- 172.21.15.37:0/4150404640 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f23f0062570 con 0x7f23f40d27c0 2024-09-17T04:33:40.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.707+0000 7f23fbf58640 1 -- 172.21.15.37:0/4150404640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23f40d2280 con 0x7f23f40d27c0 2024-09-17T04:33:40.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.708+0000 7f23ea7fc640 1 -- 172.21.15.37:0/4150404640 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16605 (secure 0 0 0) 0x7f23f0066fa0 con 0x7f23f40d27c0 2024-09-17T04:33:40.708 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:40.708 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":22,"fsid":"a908a603-430c-4209-8294-0c3460b35f42","created":"2024-09-17T04:32:54.139221+0000","modified":"2024-09-17T04:33:20.677493+0000","last_up_change":"2024-09-17T04:33:06.922210+0000","last_in_change":"2024-09-17T04:32:57.754118+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:33:08.461907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:33:16.958785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:18.013768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ea4b8d1-0d22-4dd9-b913-9df585ef24ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6809","nonce":2283587593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6811","nonce":2283587593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6815","nonce":2283587593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6813","nonce":2283587593}]},"public_addr":"172.21.15.37:6809/2283587593","cluster_addr":"172.21.15.37:6811/2283587593","heartbeat_back_addr":"172.21.15.37:6815/2283587593","heartbeat_front_addr":"172.21.15.37:6813/2283587593","state":["exists","up"]},{"osd":1,"uuid":"ac745a33-e8ad-4083-9c18-71bf81982b76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6825","nonce":2657955935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6827","nonce":2657955935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6831","nonce":2657955935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6829","nonce":2657955935}]},"public_addr":"172.21.15.37:6825/2657955935","cluster_addr":"172.21.15.37:6827/2657955935","heartbeat_back_addr":"172.21.15.37:6831/2657955935","heartbeat_front_addr":"172.21.15.37:6829/2657955935","state":["exists","up"]},{"osd":2,"uuid":"3c63f3d8-132a-4735-b0bc-d64313a46503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6817","nonce":579720176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6819","nonce":579720176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6823","nonce":579720176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6821","nonce":579720176}]},"public_addr":"172.21.15.37:6817/579720176","cluster_addr":"172.21.15.37:6819/579720176","heartbeat_back_addr":"172.21.15.37:6823/579720176","heartbeat_front_addr":"172.21.15.37:6821/579720176","state":["exists","up"]},{"osd":3,"uuid":"4f8c6ca2-ea1d-4610-9127-52c4dd8bc8e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6801","nonce":809975826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6803","nonce":809975826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6807","nonce":809975826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6805","nonce":809975826}]},"public_addr":"172.21.15.37:6801/809975826","cluster_addr":"172.21.15.37:6803/809975826","heartbeat_back_addr":"172.21.15.37:6807/809975826","heartbeat_front_addr":"172.21.15.37:6805/809975826","state":["exists","up"]},{"osd":4,"uuid":"63d0c4ab-f115-47b4-9611-12a10153473a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6819","nonce":2509785752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6821","nonce":2509785752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6825","nonce":2509785752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6823","nonce":2509785752}]},"public_addr":"172.21.15.92:6819/2509785752","cluster_addr":"172.21.15.92:6821/2509785752","heartbeat_back_addr":"172.21.15.92:6825/2509785752","heartbeat_front_addr":"172.21.15.92:6823/2509785752","state":["exists","up"]},{"osd":5,"uuid":"4c56c04d-0c07-4f76-9d75-6335121310ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6827","nonce":248263832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6829","nonce":248263832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6833","nonce":248263832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6831","nonce":248263832}]},"public_addr":"172.21.15.92:6827/248263832","cluster_addr":"172.21.15.92:6829/248263832","heartbeat_back_addr":"172.21.15.92:6833/248263832","heartbeat_front_addr":"172.21.15.92:6831/248263832","state":["exists","up"]},{"osd":6,"uuid":"b30eae7c-096a-4d24-ac51-96b8b13ecaee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6811","nonce":4078365985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6813","nonce":4078365985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6817","nonce":4078365985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6815","nonce":4078365985}]},"public_addr":"172.21.15.92:6811/4078365985","cluster_addr":"172.21.15.92:6813/4078365985","heartbeat_back_addr":"172.21.15.92:6817/4078365985","heartbeat_front_addr":"172.21.15.92:6815/4078365985","state":["exists","up"]},{"osd":7,"uuid":"62f12540-d374-4a4c-9ff7-63b7304fe77f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6803","nonce":3444787223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6805","nonce":3444787223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6809","nonce":3444787223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6807","nonce":3444787223}]},"public_addr":"172.21.15.92:6803/3444787223","cluster_addr":"172.21.15.92:6805/3444787223","heartbeat_back_addr":"172.21.15.92:6809/3444787223","heartbeat_front_addr":"172.21.15.92:6807/3444787223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.006420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.111135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.103325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.072656+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.200989+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.060455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.774773+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.865778+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:33:40.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.710+0000 7f23fbf58640 1 -- 172.21.15.37:0/4150404640 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f23d8078b90 msgr2=0x7f23d807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:40.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.710+0000 7f23fbf58640 1 --2- 172.21.15.37:0/4150404640 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f23d8078b90 0x7f23d807b0a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f23e4034000 tx=0x7f23e4004730 comp rx=0 tx=0).stop 2024-09-17T04:33:40.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.710+0000 7f23fbf58640 1 -- 172.21.15.37:0/4150404640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f40d27c0 msgr2=0x7f23f40b6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:40.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.710+0000 7f23fbf58640 1 --2- 172.21.15.37:0/4150404640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f40d27c0 0x7f23f40b6960 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f23f000c460 tx=0x7f23f000c920 comp rx=0 tx=0).stop 2024-09-17T04:33:40.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.710+0000 7f23fbf58640 1 -- 172.21.15.37:0/4150404640 shutdown_connections 2024-09-17T04:33:40.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.710+0000 7f23fbf58640 1 --2- 172.21.15.37:0/4150404640 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f23d8078b90 0x7f23d807b0a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:40.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.710+0000 7f23fbf58640 1 --2- 172.21.15.37:0/4150404640 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f23f415ff90 0x7f23f40b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:40.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.710+0000 7f23fbf58640 1 --2- 172.21.15.37:0/4150404640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23f415e7c0 0x7f23f41539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:40.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.710+0000 7f23fbf58640 1 --2- 172.21.15.37:0/4150404640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f40d27c0 0x7f23f40b6960 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:40.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.710+0000 7f23fbf58640 1 -- 172.21.15.37:0/4150404640 >> 172.21.15.37:0/4150404640 conn(0x7f23f4072a80 msgr2=0x7f23f40b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:40.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.710+0000 7f23fbf58640 1 -- 172.21.15.37:0/4150404640 shutdown_connections 2024-09-17T04:33:40.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.710+0000 7f23fbf58640 1 -- 172.21.15.37:0/4150404640 wait complete. 2024-09-17T04:33:40.721 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T04:33:08.461907+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-17T04:33:16.958785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-17T04:33:18.013768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T04:33:40.721 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T04:33:40.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.841+0000 7eff923c3640 1 -- 172.21.15.37:0/2873419801 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7eff8c163af0 msgr2=0x7eff8c165f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:40.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.841+0000 7eff923c3640 1 --2- 172.21.15.37:0/2873419801 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7eff8c163af0 0x7eff8c165f10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7eff8400acf0 tx=0x7eff8402bac0 comp rx=0 tx=0).stop 2024-09-17T04:33:40.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.841+0000 7eff923c3640 1 -- 172.21.15.37:0/2873419801 shutdown_connections 2024-09-17T04:33:40.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.841+0000 7eff923c3640 1 --2- 172.21.15.37:0/2873419801 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7eff8c163af0 0x7eff8c165f10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:40.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.841+0000 7eff923c3640 1 -- 172.21.15.37:0/2873419801 >> 172.21.15.37:0/2873419801 conn(0x7eff8c0d00b0 msgr2=0x7eff8c0d04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:40.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.841+0000 7eff923c3640 1 -- 172.21.15.37:0/2873419801 shutdown_connections 2024-09-17T04:33:40.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.841+0000 7eff923c3640 1 -- 172.21.15.37:0/2873419801 wait complete. 2024-09-17T04:33:40.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.843+0000 7eff923c3640 1 Processor -- start 2024-09-17T04:33:40.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.843+0000 7eff923c3640 1 -- start start 2024-09-17T04:33:40.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.844+0000 7eff923c3640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7eff8c06fe30 0x7eff8c14f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:40.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.844+0000 7eff923c3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff8c09dc60 0x7eff8c152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:40.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.844+0000 7eff923c3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff8c163af0 0x7eff8c14fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:40.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.844+0000 7eff923c3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c06efb0 con 0x7eff8c09dc60 2024-09-17T04:33:40.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.844+0000 7eff923c3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c06ee30 con 0x7eff8c163af0 2024-09-17T04:33:40.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.844+0000 7eff923c3640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7eff8c06f130 con 0x7eff8c06fe30 2024-09-17T04:33:40.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.844+0000 7eff8b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff8c09dc60 0x7eff8c152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:40.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.845+0000 7eff8b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff8c09dc60 0x7eff8c152520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52934/0 (socket says 172.21.15.37:52934) 2024-09-17T04:33:40.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.845+0000 7eff8b7fe640 1 -- 172.21.15.37:0/2097066454 learned_addr learned my addr 172.21.15.37:0/2097066454 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:40.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.845+0000 7eff90960640 1 --2- 172.21.15.37:0/2097066454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff8c163af0 0x7eff8c14fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:40.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.845+0000 7eff8bfff640 1 --2- 172.21.15.37:0/2097066454 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7eff8c06fe30 0x7eff8c14f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:40.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.845+0000 7eff8b7fe640 1 -- 172.21.15.37:0/2097066454 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7eff8c06fe30 msgr2=0x7eff8c14f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:40.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.845+0000 7eff8b7fe640 1 --2- 172.21.15.37:0/2097066454 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7eff8c06fe30 0x7eff8c14f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:40.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.845+0000 7eff8b7fe640 1 -- 172.21.15.37:0/2097066454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff8c163af0 msgr2=0x7eff8c14fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:40.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.845+0000 7eff8b7fe640 1 --2- 172.21.15.37:0/2097066454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff8c163af0 0x7eff8c14fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:40.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.845+0000 7eff8b7fe640 1 -- 172.21.15.37:0/2097066454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8c158950 con 0x7eff8c09dc60 2024-09-17T04:33:40.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.845+0000 7eff8b7fe640 1 --2- 172.21.15.37:0/2097066454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff8c09dc60 0x7eff8c152520 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7eff8000c430 tx=0x7eff8000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:40.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.845+0000 7eff88ff9640 1 -- 172.21.15.37:0/2097066454 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff80017070 con 0x7eff8c09dc60 2024-09-17T04:33:40.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.845+0000 7eff88ff9640 1 -- 172.21.15.37:0/2097066454 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff800053e0 con 0x7eff8c09dc60 2024-09-17T04:33:40.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.845+0000 7eff88ff9640 1 -- 172.21.15.37:0/2097066454 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff80003850 con 0x7eff8c09dc60 2024-09-17T04:33:40.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.845+0000 7eff923c3640 1 -- 172.21.15.37:0/2097066454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff8c158670 con 0x7eff8c09dc60 2024-09-17T04:33:40.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.845+0000 7eff923c3640 1 -- 172.21.15.37:0/2097066454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff8c0b3390 con 0x7eff8c09dc60 2024-09-17T04:33:40.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.847+0000 7eff923c3640 1 -- 172.21.15.37:0/2097066454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff8c070240 con 0x7eff8c09dc60 2024-09-17T04:33:40.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.847+0000 7eff88ff9640 1 -- 172.21.15.37:0/2097066454 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7eff8001a070 con 0x7eff8c09dc60 2024-09-17T04:33:40.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.847+0000 7eff88ff9640 1 --2- 172.21.15.37:0/2097066454 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7eff78078bc0 0x7eff7807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:40.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.848+0000 7eff88ff9640 1 -- 172.21.15.37:0/2097066454 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7eff80099fa0 con 0x7eff8c09dc60 2024-09-17T04:33:40.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.848+0000 7eff8bfff640 1 --2- 172.21.15.37:0/2097066454 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7eff78078bc0 0x7eff7807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:40.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.848+0000 7eff8bfff640 1 --2- 172.21.15.37:0/2097066454 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7eff78078bc0 0x7eff7807b0d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7eff7c001860 tx=0x7eff7c0016d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:40.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:40.852+0000 7eff88ff9640 1 -- 172.21.15.37:0/2097066454 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7eff80061e20 con 0x7eff8c09dc60 2024-09-17T04:33:41.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.041+0000 7eff923c3640 1 -- 172.21.15.37:0/2097066454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7eff8c06a110 con 0x7eff8c09dc60 2024-09-17T04:33:41.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.042+0000 7eff88ff9640 1 -- 172.21.15.37:0/2097066454 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7eff80066850 con 0x7eff8c09dc60 2024-09-17T04:33:41.042 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-09-17T04:33:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.044+0000 7eff923c3640 1 -- 172.21.15.37:0/2097066454 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7eff78078bc0 msgr2=0x7eff7807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.044+0000 7eff923c3640 1 --2- 172.21.15.37:0/2097066454 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7eff78078bc0 0x7eff7807b0d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7eff7c001860 tx=0x7eff7c0016d0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.044+0000 7eff923c3640 1 -- 172.21.15.37:0/2097066454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff8c09dc60 msgr2=0x7eff8c152520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.044+0000 7eff923c3640 1 --2- 172.21.15.37:0/2097066454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff8c09dc60 0x7eff8c152520 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7eff8000c430 tx=0x7eff8000c8f0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.044+0000 7eff923c3640 1 -- 172.21.15.37:0/2097066454 shutdown_connections 2024-09-17T04:33:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.044+0000 7eff923c3640 1 --2- 172.21.15.37:0/2097066454 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7eff78078bc0 0x7eff7807b0d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.044+0000 7eff923c3640 1 --2- 172.21.15.37:0/2097066454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff8c163af0 0x7eff8c14fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.044+0000 7eff923c3640 1 --2- 172.21.15.37:0/2097066454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff8c09dc60 0x7eff8c152520 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.044+0000 7eff923c3640 1 --2- 172.21.15.37:0/2097066454 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7eff8c06fe30 0x7eff8c14f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.044+0000 7eff923c3640 1 -- 172.21.15.37:0/2097066454 >> 172.21.15.37:0/2097066454 conn(0x7eff8c0d00b0 msgr2=0x7eff8c153950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:41.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.044+0000 7eff923c3640 1 -- 172.21.15.37:0/2097066454 shutdown_connections 2024-09-17T04:33:41.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.044+0000 7eff923c3640 1 -- 172.21.15.37:0/2097066454 wait complete. 2024-09-17T04:33:41.055 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-17T04:33:41.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.175+0000 7fdd0224a640 1 -- 172.21.15.37:0/2802154009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcfc0cfe10 msgr2=0x7fdcfc162590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:41.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.175+0000 7fdd0224a640 1 --2- 172.21.15.37:0/2802154009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcfc0cfe10 0x7fdcfc162590 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdcec005fd0 tx=0x7fdcec02a540 comp rx=0 tx=0).stop 2024-09-17T04:33:41.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.175+0000 7fdd0224a640 1 -- 172.21.15.37:0/2802154009 shutdown_connections 2024-09-17T04:33:41.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.175+0000 7fdd0224a640 1 --2- 172.21.15.37:0/2802154009 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdcfc15eba0 0x7fdcfc0d2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.175+0000 7fdd0224a640 1 --2- 172.21.15.37:0/2802154009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcfc0cfe10 0x7fdcfc162590 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.175+0000 7fdd0224a640 1 -- 172.21.15.37:0/2802154009 >> 172.21.15.37:0/2802154009 conn(0x7fdcfc06ddb0 msgr2=0x7fdcfc06e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:41.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.175+0000 7fdd0224a640 1 -- 172.21.15.37:0/2802154009 shutdown_connections 2024-09-17T04:33:41.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.175+0000 7fdd0224a640 1 -- 172.21.15.37:0/2802154009 wait complete. 2024-09-17T04:33:41.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.177+0000 7fdd0224a640 1 Processor -- start 2024-09-17T04:33:41.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.177+0000 7fdd0224a640 1 -- start start 2024-09-17T04:33:41.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.179+0000 7fdd0224a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcfc0cfe10 0x7fdcfc1f5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:41.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.179+0000 7fdd0224a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcfc15e790 0x7fdcfc0cb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:41.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.179+0000 7fdd0224a640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdcfc15eba0 0x7fdcfc0b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:41.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.179+0000 7fdd0224a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdcfc0d38c0 con 0x7fdcfc0cfe10 2024-09-17T04:33:41.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.179+0000 7fdd0224a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdcfc0d3740 con 0x7fdcfc15e790 2024-09-17T04:33:41.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.179+0000 7fdd0224a640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fdcfc0d3a40 con 0x7fdcfc15eba0 2024-09-17T04:33:41.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.179+0000 7fdcfb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcfc0cfe10 0x7fdcfc1f5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:41.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.179+0000 7fdcfb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcfc0cfe10 0x7fdcfc1f5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52952/0 (socket says 172.21.15.37:52952) 2024-09-17T04:33:41.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.179+0000 7fdcfb7fe640 1 -- 172.21.15.37:0/1023209830 learned_addr learned my addr 172.21.15.37:0/1023209830 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:41.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.179+0000 7fdcfaffd640 1 --2- 172.21.15.37:0/1023209830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcfc15e790 0x7fdcfc0cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:41.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.179+0000 7fdcfbfff640 1 --2- 172.21.15.37:0/1023209830 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdcfc15eba0 0x7fdcfc0b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:41.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.179+0000 7fdcfb7fe640 1 -- 172.21.15.37:0/1023209830 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdcfc15eba0 msgr2=0x7fdcfc0b60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:41.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.179+0000 7fdcfb7fe640 1 --2- 172.21.15.37:0/1023209830 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdcfc15eba0 0x7fdcfc0b60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.179+0000 7fdcfb7fe640 1 -- 172.21.15.37:0/1023209830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcfc15e790 msgr2=0x7fdcfc0cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.179+0000 7fdcfb7fe640 1 --2- 172.21.15.37:0/1023209830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcfc15e790 0x7fdcfc0cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.179+0000 7fdcfb7fe640 1 -- 172.21.15.37:0/1023209830 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcfc0cc4a0 con 0x7fdcfc0cfe10 2024-09-17T04:33:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.179+0000 7fdcfb7fe640 1 --2- 172.21.15.37:0/1023209830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcfc0cfe10 0x7fdcfc1f5ee0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdcec006010 tx=0x7fdcec0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.179+0000 7fdce3fff640 1 -- 172.21.15.37:0/1023209830 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec03e030 con 0x7fdcfc0cfe10 2024-09-17T04:33:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.179+0000 7fdce3fff640 1 -- 172.21.15.37:0/1023209830 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcec003d90 con 0x7fdcfc0cfe10 2024-09-17T04:33:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.179+0000 7fdd0224a640 1 -- 172.21.15.37:0/1023209830 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcfc0cc1c0 con 0x7fdcfc0cfe10 2024-09-17T04:33:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.179+0000 7fdce3fff640 1 -- 172.21.15.37:0/1023209830 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec035040 con 0x7fdcfc0cfe10 2024-09-17T04:33:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.180+0000 7fdd0224a640 1 -- 172.21.15.37:0/1023209830 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcfc0cd160 con 0x7fdcfc0cfe10 2024-09-17T04:33:41.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.180+0000 7fdd0224a640 1 -- 172.21.15.37:0/1023209830 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcfc0d3f60 con 0x7fdcfc0cfe10 2024-09-17T04:33:41.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.181+0000 7fdce3fff640 1 -- 172.21.15.37:0/1023209830 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdcec041030 con 0x7fdcfc0cfe10 2024-09-17T04:33:41.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.182+0000 7fdce3fff640 1 --2- 172.21.15.37:0/1023209830 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fdce8078bc0 0x7fdce807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:41.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.182+0000 7fdce3fff640 1 -- 172.21.15.37:0/1023209830 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fdcec0bb370 con 0x7fdcfc0cfe10 2024-09-17T04:33:41.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.182+0000 7fdcfaffd640 1 --2- 172.21.15.37:0/1023209830 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fdce8078bc0 0x7fdce807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:41.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.183+0000 7fdcfaffd640 1 --2- 172.21.15.37:0/1023209830 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fdce8078bc0 0x7fdce807b0d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0cf7c0 tx=0x7fdcf0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:41.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.187+0000 7fdce3fff640 1 -- 172.21.15.37:0/1023209830 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fdcec083160 con 0x7fdcfc0cfe10 2024-09-17T04:33:41.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.370+0000 7fdd0224a640 1 -- 172.21.15.37:0/1023209830 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdcfc1f6420 con 0x7fdcfc0cfe10 2024-09-17T04:33:41.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.371+0000 7fdce3fff640 1 -- 172.21.15.37:0/1023209830 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fdcec087b90 con 0x7fdcfc0cfe10 2024-09-17T04:33:41.371 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-09-17T04:33:41.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.373+0000 7fdd0224a640 1 -- 172.21.15.37:0/1023209830 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fdce8078bc0 msgr2=0x7fdce807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:41.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.373+0000 7fdd0224a640 1 --2- 172.21.15.37:0/1023209830 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fdce8078bc0 0x7fdce807b0d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0cf7c0 tx=0x7fdcf0009040 comp rx=0 tx=0).stop 2024-09-17T04:33:41.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.373+0000 7fdd0224a640 1 -- 172.21.15.37:0/1023209830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcfc0cfe10 msgr2=0x7fdcfc1f5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:41.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.373+0000 7fdd0224a640 1 --2- 172.21.15.37:0/1023209830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcfc0cfe10 0x7fdcfc1f5ee0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdcec006010 tx=0x7fdcec0038a0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.373+0000 7fdd0224a640 1 -- 172.21.15.37:0/1023209830 shutdown_connections 2024-09-17T04:33:41.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.373+0000 7fdd0224a640 1 --2- 172.21.15.37:0/1023209830 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fdce8078bc0 0x7fdce807b0d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.373+0000 7fdd0224a640 1 --2- 172.21.15.37:0/1023209830 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdcfc15eba0 0x7fdcfc0b60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.373+0000 7fdd0224a640 1 --2- 172.21.15.37:0/1023209830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcfc15e790 0x7fdcfc0cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.373+0000 7fdd0224a640 1 --2- 172.21.15.37:0/1023209830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcfc0cfe10 0x7fdcfc1f5ee0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.373+0000 7fdd0224a640 1 -- 172.21.15.37:0/1023209830 >> 172.21.15.37:0/1023209830 conn(0x7fdcfc06ddb0 msgr2=0x7fdcfc06e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:41.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.373+0000 7fdd0224a640 1 -- 172.21.15.37:0/1023209830 shutdown_connections 2024-09-17T04:33:41.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.373+0000 7fdd0224a640 1 -- 172.21.15.37:0/1023209830 wait complete. 2024-09-17T04:33:41.384 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-17T04:33:41.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.504+0000 7f8ff27ba640 1 -- 172.21.15.37:0/410282293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec162590 msgr2=0x7f8fec0d0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:41.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.504+0000 7f8ff27ba640 1 --2- 172.21.15.37:0/410282293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec162590 0x7f8fec0d0c40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0d29c0 tx=0x7f8fe002a4a0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.504+0000 7f8ff27ba640 1 -- 172.21.15.37:0/410282293 shutdown_connections 2024-09-17T04:33:41.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.504+0000 7f8ff27ba640 1 --2- 172.21.15.37:0/410282293 >> v2:172.21.15.92:3300/0 conn(0x7f8fec0d1180 0x7f8fec0d2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.504+0000 7f8ff27ba640 1 --2- 172.21.15.37:0/410282293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec162590 0x7f8fec0d0c40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.504+0000 7f8ff27ba640 1 -- 172.21.15.37:0/410282293 >> 172.21.15.37:0/410282293 conn(0x7f8fec06ddb0 msgr2=0x7f8fec06e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:41.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.504+0000 7f8ff27ba640 1 -- 172.21.15.37:0/410282293 shutdown_connections 2024-09-17T04:33:41.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.504+0000 7f8ff27ba640 1 -- 172.21.15.37:0/410282293 wait complete. 2024-09-17T04:33:41.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.506+0000 7f8ff27ba640 1 Processor -- start 2024-09-17T04:33:41.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.506+0000 7f8ff27ba640 1 -- start start 2024-09-17T04:33:41.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.507+0000 7f8ff27ba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec0cfe10 0x7f8fec0b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:41.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.507+0000 7f8ff27ba640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fec0d1180 0x7f8fec1f9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:41.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.507+0000 7f8ff27ba640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8fec162590 0x7f8fec14d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:41.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.507+0000 7f8ff27ba640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8fec0d3810 con 0x7f8fec0cfe10 2024-09-17T04:33:41.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.507+0000 7f8ff27ba640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8fec0d3690 con 0x7f8fec0d1180 2024-09-17T04:33:41.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.507+0000 7f8ff27ba640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f8fec0d3990 con 0x7f8fec162590 2024-09-17T04:33:41.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.507+0000 7f8febfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec0cfe10 0x7f8fec0b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:41.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.507+0000 7f8febfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec0cfe10 0x7f8fec0b69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52970/0 (socket says 172.21.15.37:52970) 2024-09-17T04:33:41.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.507+0000 7f8febfff640 1 -- 172.21.15.37:0/3087172465 learned_addr learned my addr 172.21.15.37:0/3087172465 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:41.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.508+0000 7f8feb7fe640 1 --2- 172.21.15.37:0/3087172465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fec0d1180 0x7f8fec1f9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:41.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.508+0000 7f8ff0d57640 1 --2- 172.21.15.37:0/3087172465 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8fec162590 0x7f8fec14d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:41.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.508+0000 7f8febfff640 1 -- 172.21.15.37:0/3087172465 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8fec162590 msgr2=0x7f8fec14d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:41.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.508+0000 7f8febfff640 1 --2- 172.21.15.37:0/3087172465 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8fec162590 0x7f8fec14d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.508+0000 7f8febfff640 1 -- 172.21.15.37:0/3087172465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fec0d1180 msgr2=0x7f8fec1f9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:41.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.508+0000 7f8febfff640 1 --2- 172.21.15.37:0/3087172465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fec0d1180 0x7f8fec1f9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.508+0000 7f8febfff640 1 -- 172.21.15.37:0/3087172465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fec14de10 con 0x7f8fec0cfe10 2024-09-17T04:33:41.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.508+0000 7f8feb7fe640 1 --2- 172.21.15.37:0/3087172465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fec0d1180 0x7f8fec1f9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:41.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.508+0000 7f8ff0d57640 1 --2- 172.21.15.37:0/3087172465 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8fec162590 0x7f8fec14d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:41.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.508+0000 7f8febfff640 1 --2- 172.21.15.37:0/3087172465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec0cfe10 0x7f8fec0b69f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc00c3a0 tx=0x7f8fdc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:41.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.508+0000 7f8fe8ff9640 1 -- 172.21.15.37:0/3087172465 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc011790 con 0x7f8fec0cfe10 2024-09-17T04:33:41.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.508+0000 7f8fe8ff9640 1 -- 172.21.15.37:0/3087172465 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fdc017070 con 0x7f8fec0cfe10 2024-09-17T04:33:41.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.508+0000 7f8ff27ba640 1 -- 172.21.15.37:0/3087172465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fec0b5950 con 0x7f8fec0cfe10 2024-09-17T04:33:41.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.508+0000 7f8fe8ff9640 1 -- 172.21.15.37:0/3087172465 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc011cc0 con 0x7f8fec0cfe10 2024-09-17T04:33:41.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.508+0000 7f8ff27ba640 1 -- 172.21.15.37:0/3087172465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fec0b5f10 con 0x7f8fec0cfe10 2024-09-17T04:33:41.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.509+0000 7f8ff27ba640 1 -- 172.21.15.37:0/3087172465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fec160190 con 0x7f8fec0cfe10 2024-09-17T04:33:41.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.510+0000 7f8fe8ff9640 1 -- 172.21.15.37:0/3087172465 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8fdc01c460 con 0x7f8fec0cfe10 2024-09-17T04:33:41.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.511+0000 7f8fe8ff9640 1 --2- 172.21.15.37:0/3087172465 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8fd8078b90 0x7f8fd807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:41.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.511+0000 7f8fe8ff9640 1 -- 172.21.15.37:0/3087172465 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8fdc09ab00 con 0x7f8fec0cfe10 2024-09-17T04:33:41.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.511+0000 7f8feb7fe640 1 --2- 172.21.15.37:0/3087172465 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8fd8078b90 0x7f8fd807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:41.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.512+0000 7f8feb7fe640 1 --2- 172.21.15.37:0/3087172465 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8fd8078b90 0x7f8fd807b0a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0034000 tx=0x7f8fe0004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:41.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.515+0000 7f8fe8ff9640 1 -- 172.21.15.37:0/3087172465 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8fec160190 con 0x7f8fec0cfe10 2024-09-17T04:33:41.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.703+0000 7f8ff27ba640 1 -- 172.21.15.37:0/3087172465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8fec0d3eb0 con 0x7f8fec0cfe10 2024-09-17T04:33:41.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.704+0000 7f8fe8ff9640 1 -- 172.21.15.37:0/3087172465 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f8fdc062f50 con 0x7f8fec0cfe10 2024-09-17T04:33:41.704 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-09-17T04:33:41.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.706+0000 7f8ff27ba640 1 -- 172.21.15.37:0/3087172465 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8fd8078b90 msgr2=0x7f8fd807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:41.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.706+0000 7f8ff27ba640 1 --2- 172.21.15.37:0/3087172465 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8fd8078b90 0x7f8fd807b0a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0034000 tx=0x7f8fe0004730 comp rx=0 tx=0).stop 2024-09-17T04:33:41.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.706+0000 7f8ff27ba640 1 -- 172.21.15.37:0/3087172465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec0cfe10 msgr2=0x7f8fec0b69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:41.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.706+0000 7f8ff27ba640 1 --2- 172.21.15.37:0/3087172465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec0cfe10 0x7f8fec0b69f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc00c3a0 tx=0x7f8fdc00c860 comp rx=0 tx=0).stop 2024-09-17T04:33:41.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.706+0000 7f8ff27ba640 1 -- 172.21.15.37:0/3087172465 shutdown_connections 2024-09-17T04:33:41.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.706+0000 7f8ff27ba640 1 --2- 172.21.15.37:0/3087172465 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8fd8078b90 0x7f8fd807b0a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.706+0000 7f8ff27ba640 1 --2- 172.21.15.37:0/3087172465 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8fec162590 0x7f8fec14d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.706+0000 7f8ff27ba640 1 --2- 172.21.15.37:0/3087172465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fec0d1180 0x7f8fec1f9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.706+0000 7f8ff27ba640 1 --2- 172.21.15.37:0/3087172465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec0cfe10 0x7f8fec0b69f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.706+0000 7f8ff27ba640 1 -- 172.21.15.37:0/3087172465 >> 172.21.15.37:0/3087172465 conn(0x7f8fec06ddb0 msgr2=0x7f8fec15fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:41.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.706+0000 7f8ff27ba640 1 -- 172.21.15.37:0/3087172465 shutdown_connections 2024-09-17T04:33:41.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.706+0000 7f8ff27ba640 1 -- 172.21.15.37:0/3087172465 wait complete. 2024-09-17T04:33:41.717 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_recovery_pool.TestRecoveryPool.test_rebuild_simple' 2024-09-17T04:33:41.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.834+0000 7fa2de907640 1 -- 172.21.15.37:0/3370436680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d815c630 msgr2=0x7fa2d80cff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:41.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.834+0000 7fa2de907640 1 --2- 172.21.15.37:0/3370436680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d815c630 0x7fa2d80cff30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa2d80d1780 tx=0x7fa2c802a4f0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.834+0000 7fa2de907640 1 -- 172.21.15.37:0/3370436680 shutdown_connections 2024-09-17T04:33:41.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.834+0000 7fa2de907640 1 --2- 172.21.15.37:0/3370436680 >> v2:172.21.15.92:3300/0 conn(0x7fa2d80ccbc0 0x7fa2d80ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.834+0000 7fa2de907640 1 --2- 172.21.15.37:0/3370436680 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa2d80d19c0 0x7fa2d80d1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.834+0000 7fa2de907640 1 --2- 172.21.15.37:0/3370436680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d815c630 0x7fa2d80cff30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.834+0000 7fa2de907640 1 -- 172.21.15.37:0/3370436680 >> 172.21.15.37:0/3370436680 conn(0x7fa2d80cede0 msgr2=0x7fa2d80cf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:41.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.834+0000 7fa2de907640 1 -- 172.21.15.37:0/3370436680 shutdown_connections 2024-09-17T04:33:41.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.834+0000 7fa2de907640 1 -- 172.21.15.37:0/3370436680 wait complete. 2024-09-17T04:33:41.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.836+0000 7fa2de907640 1 Processor -- start 2024-09-17T04:33:41.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.836+0000 7fa2de907640 1 -- start start 2024-09-17T04:33:41.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.838+0000 7fa2de907640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d80ccbc0 0x7fa2d806f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:41.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.838+0000 7fa2de907640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d80d19c0 0x7fa2d81577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:41.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.838+0000 7fa2de907640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa2d815c630 0x7fa2d81f7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:41.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.838+0000 7fa2de907640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d815d0c0 con 0x7fa2d80d19c0 2024-09-17T04:33:41.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.838+0000 7fa2de907640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81602a0 con 0x7fa2d80ccbc0 2024-09-17T04:33:41.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.838+0000 7fa2de907640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d815d240 con 0x7fa2d815c630 2024-09-17T04:33:41.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.838+0000 7fa2d77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d80d19c0 0x7fa2d81577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:41.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.838+0000 7fa2d77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d80d19c0 0x7fa2d81577a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52974/0 (socket says 172.21.15.37:52974) 2024-09-17T04:33:41.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.838+0000 7fa2d77fe640 1 -- 172.21.15.37:0/3091674124 learned_addr learned my addr 172.21.15.37:0/3091674124 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:41.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.838+0000 7fa2dcea4640 1 --2- 172.21.15.37:0/3091674124 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa2d815c630 0x7fa2d81f7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:41.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.838+0000 7fa2d7fff640 1 --2- 172.21.15.37:0/3091674124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d80ccbc0 0x7fa2d806f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:41.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.838+0000 7fa2d77fe640 1 -- 172.21.15.37:0/3091674124 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa2d815c630 msgr2=0x7fa2d81f7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.838+0000 7fa2d77fe640 1 --2- 172.21.15.37:0/3091674124 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa2d815c630 0x7fa2d81f7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.838+0000 7fa2d77fe640 1 -- 172.21.15.37:0/3091674124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d80ccbc0 msgr2=0x7fa2d806f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.838+0000 7fa2d77fe640 1 --2- 172.21.15.37:0/3091674124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d80ccbc0 0x7fa2d806f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.838+0000 7fa2d77fe640 1 -- 172.21.15.37:0/3091674124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d806e1c0 con 0x7fa2d80d19c0 2024-09-17T04:33:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.838+0000 7fa2d77fe640 1 --2- 172.21.15.37:0/3091674124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d80d19c0 0x7fa2d81577a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00c970 tx=0x7fa2cc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.838+0000 7fa2d4ff9640 1 -- 172.21.15.37:0/3091674124 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc007bb0 con 0x7fa2d80d19c0 2024-09-17T04:33:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.839+0000 7fa2d4ff9640 1 -- 172.21.15.37:0/3091674124 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2cc007e40 con 0x7fa2d80d19c0 2024-09-17T04:33:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.839+0000 7fa2de907640 1 -- 172.21.15.37:0/3091674124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d806e4a0 con 0x7fa2d80d19c0 2024-09-17T04:33:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.839+0000 7fa2d4ff9640 1 -- 172.21.15.37:0/3091674124 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc00f6b0 con 0x7fa2d80d19c0 2024-09-17T04:33:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.839+0000 7fa2de907640 1 -- 172.21.15.37:0/3091674124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d806ea60 con 0x7fa2d80d19c0 2024-09-17T04:33:41.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.840+0000 7fa2de907640 1 -- 172.21.15.37:0/3091674124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2d815d7d0 con 0x7fa2d80d19c0 2024-09-17T04:33:41.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.840+0000 7fa2d4ff9640 1 -- 172.21.15.37:0/3091674124 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa2cc018400 con 0x7fa2d80d19c0 2024-09-17T04:33:41.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.841+0000 7fa2d4ff9640 1 --2- 172.21.15.37:0/3091674124 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa2c4078b90 0x7fa2c407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:41.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.841+0000 7fa2d4ff9640 1 -- 172.21.15.37:0/3091674124 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa2cc09b300 con 0x7fa2d80d19c0 2024-09-17T04:33:41.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.841+0000 7fa2d7fff640 1 --2- 172.21.15.37:0/3091674124 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa2c4078b90 0x7fa2c407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.842+0000 7fa2d7fff640 1 --2- 172.21.15.37:0/3091674124 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa2c4078b90 0x7fa2c407b0a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8034000 tx=0x7fa2c8004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:41.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:41.846+0000 7fa2d4ff9640 1 -- 172.21.15.37:0/3091674124 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa2cc014030 con 0x7fa2d80d19c0 2024-09-17T04:33:42.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:42.104+0000 7fa2de907640 1 -- 172.21.15.37:0/3091674124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_recovery_pool.TestRecoveryPool.test_rebuild_simple"]} v 0) -- 0x7fa2d8073550 con 0x7fa2d80d19c0 2024-09-17T04:33:42.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:42.980+0000 7fa2d4ff9640 1 -- 172.21.15.37:0/3091674124 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_recovery_pool.TestRecoveryPool.test_rebuild_simple"]}]=0 v47) ==== 150+0+0 (secure 0 0 0) 0x7fa2cc063180 con 0x7fa2d80d19c0 2024-09-17T04:33:42.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:42.984+0000 7fa2de907640 1 -- 172.21.15.37:0/3091674124 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa2c4078b90 msgr2=0x7fa2c407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:42.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:42.984+0000 7fa2de907640 1 --2- 172.21.15.37:0/3091674124 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa2c4078b90 0x7fa2c407b0a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8034000 tx=0x7fa2c8004730 comp rx=0 tx=0).stop 2024-09-17T04:33:42.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:42.984+0000 7fa2de907640 1 -- 172.21.15.37:0/3091674124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d80d19c0 msgr2=0x7fa2d81577a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:42.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:42.984+0000 7fa2de907640 1 --2- 172.21.15.37:0/3091674124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d80d19c0 0x7fa2d81577a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00c970 tx=0x7fa2cc00ce30 comp rx=0 tx=0).stop 2024-09-17T04:33:42.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:42.985+0000 7fa2de907640 1 -- 172.21.15.37:0/3091674124 shutdown_connections 2024-09-17T04:33:42.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:42.985+0000 7fa2de907640 1 --2- 172.21.15.37:0/3091674124 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa2c4078b90 0x7fa2c407b0a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:42.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:42.985+0000 7fa2de907640 1 --2- 172.21.15.37:0/3091674124 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa2d815c630 0x7fa2d81f7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:42.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:42.985+0000 7fa2de907640 1 --2- 172.21.15.37:0/3091674124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d80d19c0 0x7fa2d81577a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:42.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:42.985+0000 7fa2de907640 1 --2- 172.21.15.37:0/3091674124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2d80ccbc0 0x7fa2d806f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:42.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:42.985+0000 7fa2de907640 1 -- 172.21.15.37:0/3091674124 >> 172.21.15.37:0/3091674124 conn(0x7fa2d80cede0 msgr2=0x7fa2d80d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:42.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:42.985+0000 7fa2de907640 1 -- 172.21.15.37:0/3091674124 shutdown_connections 2024-09-17T04:33:42.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:42.985+0000 7fa2de907640 1 -- 172.21.15.37:0/3091674124 wait complete. 2024-09-17T04:33:42.998 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T04:33:43.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.122+0000 7fcf6404a640 1 -- 172.21.15.37:0/2348541365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf5c15bc70 msgr2=0x7fcf5c15c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:43.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.122+0000 7fcf6404a640 1 --2- 172.21.15.37:0/2348541365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf5c15bc70 0x7fcf5c15c080 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0cd250 tx=0x7fcf5402a940 comp rx=0 tx=0).stop 2024-09-17T04:33:43.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.122+0000 7fcf6404a640 1 -- 172.21.15.37:0/2348541365 shutdown_connections 2024-09-17T04:33:43.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.122+0000 7fcf6404a640 1 --2- 172.21.15.37:0/2348541365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf5c15bc70 0x7fcf5c15c080 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.122+0000 7fcf6404a640 1 --2- 172.21.15.37:0/2348541365 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcf5c0cf8f0 0x7fcf5c15b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.122+0000 7fcf6404a640 1 -- 172.21.15.37:0/2348541365 >> 172.21.15.37:0/2348541365 conn(0x7fcf5c069120 msgr2=0x7fcf5c069530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:43.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.122+0000 7fcf6404a640 1 -- 172.21.15.37:0/2348541365 shutdown_connections 2024-09-17T04:33:43.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.122+0000 7fcf6404a640 1 -- 172.21.15.37:0/2348541365 wait complete. 2024-09-17T04:33:43.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.124+0000 7fcf6404a640 1 Processor -- start 2024-09-17T04:33:43.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.124+0000 7fcf6404a640 1 -- start start 2024-09-17T04:33:43.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.125+0000 7fcf6404a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf5c15bc70 0x7fcf5c1e8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:43.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.125+0000 7fcf6404a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf5c15d960 0x7fcf5c0d1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:43.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.125+0000 7fcf6404a640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcf5c0d1a60 0x7fcf5c143530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:43.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.125+0000 7fcf6404a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c0cd5e0 con 0x7fcf5c15bc70 2024-09-17T04:33:43.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.125+0000 7fcf6404a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c0cd460 con 0x7fcf5c15d960 2024-09-17T04:33:43.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.125+0000 7fcf6404a640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fcf5c15c5c0 con 0x7fcf5c0d1a60 2024-09-17T04:33:43.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.125+0000 7fcf61de6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf5c15bc70 0x7fcf5c1e8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:43.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.125+0000 7fcf61de6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf5c15bc70 0x7fcf5c1e8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52998/0 (socket says 172.21.15.37:52998) 2024-09-17T04:33:43.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.125+0000 7fcf61de6640 1 -- 172.21.15.37:0/1598469920 learned_addr learned my addr 172.21.15.37:0/1598469920 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:43.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.126+0000 7fcf625e7640 1 --2- 172.21.15.37:0/1598469920 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcf5c0d1a60 0x7fcf5c143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:43.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.126+0000 7fcf615e5640 1 --2- 172.21.15.37:0/1598469920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf5c15d960 0x7fcf5c0d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:43.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.126+0000 7fcf61de6640 1 -- 172.21.15.37:0/1598469920 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcf5c0d1a60 msgr2=0x7fcf5c143530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:43.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.126+0000 7fcf61de6640 1 --2- 172.21.15.37:0/1598469920 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcf5c0d1a60 0x7fcf5c143530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.126+0000 7fcf61de6640 1 -- 172.21.15.37:0/1598469920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf5c15d960 msgr2=0x7fcf5c0d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:43.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.126+0000 7fcf61de6640 1 --2- 172.21.15.37:0/1598469920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf5c15d960 0x7fcf5c0d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.126+0000 7fcf61de6640 1 -- 172.21.15.37:0/1598469920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf5c1f7c30 con 0x7fcf5c15bc70 2024-09-17T04:33:43.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.126+0000 7fcf615e5640 1 --2- 172.21.15.37:0/1598469920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf5c15d960 0x7fcf5c0d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:43.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.126+0000 7fcf61de6640 1 --2- 172.21.15.37:0/1598469920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf5c15bc70 0x7fcf5c1e8700 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800b3f0 tx=0x7fcf5800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:43.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.126+0000 7fcf527fc640 1 -- 172.21.15.37:0/1598469920 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58015020 con 0x7fcf5c15bc70 2024-09-17T04:33:43.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.126+0000 7fcf527fc640 1 -- 172.21.15.37:0/1598469920 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf58011920 con 0x7fcf5c15bc70 2024-09-17T04:33:43.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.126+0000 7fcf6404a640 1 -- 172.21.15.37:0/1598469920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf5c0d1e70 con 0x7fcf5c15bc70 2024-09-17T04:33:43.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.126+0000 7fcf527fc640 1 -- 172.21.15.37:0/1598469920 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58011d70 con 0x7fcf5c15bc70 2024-09-17T04:33:43.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.126+0000 7fcf6404a640 1 -- 172.21.15.37:0/1598469920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf5c1f7150 con 0x7fcf5c15bc70 2024-09-17T04:33:43.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.127+0000 7fcf6404a640 1 -- 172.21.15.37:0/1598469920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf5c15b730 con 0x7fcf5c15bc70 2024-09-17T04:33:43.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.128+0000 7fcf527fc640 1 -- 172.21.15.37:0/1598469920 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcf58018020 con 0x7fcf5c15bc70 2024-09-17T04:33:43.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.128+0000 7fcf527fc640 1 --2- 172.21.15.37:0/1598469920 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fcf40078b90 0x7fcf4007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:43.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.129+0000 7fcf527fc640 1 -- 172.21.15.37:0/1598469920 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fcf5809ad90 con 0x7fcf5c15bc70 2024-09-17T04:33:43.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.129+0000 7fcf615e5640 1 --2- 172.21.15.37:0/1598469920 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fcf40078b90 0x7fcf4007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:43.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.129+0000 7fcf615e5640 1 --2- 172.21.15.37:0/1598469920 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fcf40078b90 0x7fcf4007b0a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c005fd0 tx=0x7fcf4c005e00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:43.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.133+0000 7fcf527fc640 1 -- 172.21.15.37:0/1598469920 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fcf5801f020 con 0x7fcf5c15bc70 2024-09-17T04:33:43.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.319+0000 7fcf6404a640 1 -- 172.21.15.37:0/1598469920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fcf5c0b2990 con 0x7fcf5c15bc70 2024-09-17T04:33:43.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.320+0000 7fcf527fc640 1 -- 172.21.15.37:0/1598469920 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v31) ==== 59+0+2294 (secure 0 0 0) 0x7fcf5c0d1e70 con 0x7fcf5c15bc70 2024-09-17T04:33:43.320 INFO:teuthology.orchestra.run.smithi037.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T04:33:43.320 INFO:teuthology.orchestra.run.smithi037.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T04:33:43.320 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced client_mount_timeout 30 2024-09-17T04:33:43.320 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced debug_client 20/20 2024-09-17T04:33:43.320 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced debug_mgr 20/20 2024-09-17T04:33:43.321 INFO:teuthology.orchestra.run.smithi037.stdout:mgr advanced debug_ms 1/1 2024-09-17T04:33:43.321 INFO:teuthology.orchestra.run.smithi037.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T04:33:43.321 INFO:teuthology.orchestra.run.smithi037.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T04:33:43.321 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3939.615108 2024-09-17T04:33:43.321 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4277.944554 2024-09-17T04:33:43.321 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4277.268934 2024-09-17T04:33:43.321 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3994.557852 2024-09-17T04:33:43.321 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 7375.739841 2024-09-17T04:33:43.321 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4452.554697 2024-09-17T04:33:43.321 INFO:teuthology.orchestra.run.smithi037.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4960.864665 2024-09-17T04:33:43.321 INFO:teuthology.orchestra.run.smithi037.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4571.297496 2024-09-17T04:33:43.322 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced debug_mds 20/20 2024-09-17T04:33:43.322 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T04:33:43.322 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced debug_ms 2/2 2024-09-17T04:33:43.322 INFO:teuthology.orchestra.run.smithi037.stdout:mds dev mds_debug_frag true 2024-09-17T04:33:43.322 INFO:teuthology.orchestra.run.smithi037.stdout:mds dev mds_debug_scatterstat true 2024-09-17T04:33:43.322 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T04:33:43.322 INFO:teuthology.orchestra.run.smithi037.stdout:mds dev mds_verify_scatter true 2024-09-17T04:33:43.322 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T04:33:43.322 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T04:33:43.322 INFO:teuthology.orchestra.run.smithi037.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T04:33:43.322 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced client_mount_timeout 600 2024-09-17T04:33:43.322 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced debug_client 20/20 2024-09-17T04:33:43.322 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced debug_ms 1/1 2024-09-17T04:33:43.323 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T04:33:43.323 INFO:teuthology.orchestra.run.smithi037.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T04:33:43.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.322+0000 7fcf6404a640 1 -- 172.21.15.37:0/1598469920 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fcf40078b90 msgr2=0x7fcf4007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:43.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.322+0000 7fcf6404a640 1 --2- 172.21.15.37:0/1598469920 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fcf40078b90 0x7fcf4007b0a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c005fd0 tx=0x7fcf4c005e00 comp rx=0 tx=0).stop 2024-09-17T04:33:43.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.322+0000 7fcf6404a640 1 -- 172.21.15.37:0/1598469920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf5c15bc70 msgr2=0x7fcf5c1e8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:43.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.322+0000 7fcf6404a640 1 --2- 172.21.15.37:0/1598469920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf5c15bc70 0x7fcf5c1e8700 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800b3f0 tx=0x7fcf5800b8b0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.322+0000 7fcf6404a640 1 -- 172.21.15.37:0/1598469920 shutdown_connections 2024-09-17T04:33:43.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.322+0000 7fcf6404a640 1 --2- 172.21.15.37:0/1598469920 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fcf40078b90 0x7fcf4007b0a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.322+0000 7fcf6404a640 1 --2- 172.21.15.37:0/1598469920 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcf5c0d1a60 0x7fcf5c143530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.322+0000 7fcf6404a640 1 --2- 172.21.15.37:0/1598469920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf5c15d960 0x7fcf5c0d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.322+0000 7fcf6404a640 1 --2- 172.21.15.37:0/1598469920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf5c15bc70 0x7fcf5c1e8700 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.322+0000 7fcf6404a640 1 -- 172.21.15.37:0/1598469920 >> 172.21.15.37:0/1598469920 conn(0x7fcf5c069120 msgr2=0x7fcf5c15f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:43.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.322+0000 7fcf6404a640 1 -- 172.21.15.37:0/1598469920 shutdown_connections 2024-09-17T04:33:43.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.322+0000 7fcf6404a640 1 -- 172.21.15.37:0/1598469920 wait complete. 2024-09-17T04:33:43.333 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T04:33:43.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.454+0000 7fd9a560e640 1 -- 172.21.15.37:0/1868253528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9a0162cf0 msgr2=0x7fd9a00d2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:43.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.454+0000 7fd9a560e640 1 --2- 172.21.15.37:0/1868253528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9a0162cf0 0x7fd9a00d2650 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd9a008cc10 tx=0x7fd99802aac0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.454+0000 7fd9a560e640 1 -- 172.21.15.37:0/1868253528 shutdown_connections 2024-09-17T04:33:43.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.454+0000 7fd9a560e640 1 --2- 172.21.15.37:0/1868253528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9a0162cf0 0x7fd9a00d2650 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.454+0000 7fd9a560e640 1 --2- 172.21.15.37:0/1868253528 >> v2:172.21.15.92:3300/0 conn(0x7fd9a00b66d0 0x7fd9a01627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.454+0000 7fd9a560e640 1 -- 172.21.15.37:0/1868253528 >> 172.21.15.37:0/1868253528 conn(0x7fd9a00d4540 msgr2=0x7fd9a00d4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:43.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.454+0000 7fd9a560e640 1 -- 172.21.15.37:0/1868253528 shutdown_connections 2024-09-17T04:33:43.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.454+0000 7fd9a560e640 1 -- 172.21.15.37:0/1868253528 wait complete. 2024-09-17T04:33:43.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.456+0000 7fd9a560e640 1 Processor -- start 2024-09-17T04:33:43.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.456+0000 7fd9a560e640 1 -- start start 2024-09-17T04:33:43.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.457+0000 7fd9a560e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd9a015e830 0x7fd9a00c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:43.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.457+0000 7fd9a560e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9a0162cf0 0x7fd9a0158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:43.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.457+0000 7fd9a560e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9a00b8ad0 0x7fd9a00b8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:43.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.457+0000 7fd9a560e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a00d3060 con 0x7fd9a0162cf0 2024-09-17T04:33:43.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.457+0000 7fd9a560e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a00d2ee0 con 0x7fd9a00b8ad0 2024-09-17T04:33:43.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.457+0000 7fd9a560e640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a00d31e0 con 0x7fd9a015e830 2024-09-17T04:33:43.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.457+0000 7fd99e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9a0162cf0 0x7fd9a0158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:43.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.457+0000 7fd99e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9a0162cf0 0x7fd9a0158130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53010/0 (socket says 172.21.15.37:53010) 2024-09-17T04:33:43.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.457+0000 7fd99e7fc640 1 -- 172.21.15.37:0/1406148643 learned_addr learned my addr 172.21.15.37:0/1406148643 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:43.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.457+0000 7fd99f7fe640 1 --2- 172.21.15.37:0/1406148643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9a00b8ad0 0x7fd9a00b8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:43.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.457+0000 7fd99effd640 1 --2- 172.21.15.37:0/1406148643 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd9a015e830 0x7fd9a00c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:43.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.457+0000 7fd99e7fc640 1 -- 172.21.15.37:0/1406148643 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd9a015e830 msgr2=0x7fd9a00c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:43.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.457+0000 7fd99e7fc640 1 --2- 172.21.15.37:0/1406148643 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd9a015e830 0x7fd9a00c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.457+0000 7fd99e7fc640 1 -- 172.21.15.37:0/1406148643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9a00b8ad0 msgr2=0x7fd9a00b8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:43.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.457+0000 7fd99e7fc640 1 --2- 172.21.15.37:0/1406148643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9a00b8ad0 0x7fd9a00b8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.457+0000 7fd99e7fc640 1 -- 172.21.15.37:0/1406148643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a00b9420 con 0x7fd9a0162cf0 2024-09-17T04:33:43.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.457+0000 7fd99effd640 1 --2- 172.21.15.37:0/1406148643 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd9a015e830 0x7fd9a00c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:43.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.458+0000 7fd99e7fc640 1 --2- 172.21.15.37:0/1406148643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9a0162cf0 0x7fd9a0158130 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd99400d6d0 tx=0x7fd99400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:43.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.458+0000 7fd9877fe640 1 -- 172.21.15.37:0/1406148643 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994017070 con 0x7fd9a0162cf0 2024-09-17T04:33:43.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.458+0000 7fd9877fe640 1 -- 172.21.15.37:0/1406148643 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd994004310 con 0x7fd9a0162cf0 2024-09-17T04:33:43.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.458+0000 7fd9a560e640 1 -- 172.21.15.37:0/1406148643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a0158670 con 0x7fd9a0162cf0 2024-09-17T04:33:43.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.458+0000 7fd9877fe640 1 -- 172.21.15.37:0/1406148643 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994009a80 con 0x7fd9a0162cf0 2024-09-17T04:33:43.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.458+0000 7fd9a560e640 1 -- 172.21.15.37:0/1406148643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a00c8210 con 0x7fd9a0162cf0 2024-09-17T04:33:43.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.459+0000 7fd9a560e640 1 -- 172.21.15.37:0/1406148643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9a00d3690 con 0x7fd9a0162cf0 2024-09-17T04:33:43.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.460+0000 7fd9877fe640 1 -- 172.21.15.37:0/1406148643 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd9940035f0 con 0x7fd9a0162cf0 2024-09-17T04:33:43.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.460+0000 7fd9877fe640 1 --2- 172.21.15.37:0/1406148643 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd98c078b90 0x7fd98c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:43.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.460+0000 7fd9877fe640 1 -- 172.21.15.37:0/1406148643 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd99409af00 con 0x7fd9a0162cf0 2024-09-17T04:33:43.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.461+0000 7fd99effd640 1 --2- 172.21.15.37:0/1406148643 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd98c078b90 0x7fd98c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:43.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.461+0000 7fd99effd640 1 --2- 172.21.15.37:0/1406148643 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd98c078b90 0x7fd98c07b0a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd990000f60 tx=0x7fd990009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:43.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.465+0000 7fd9877fe640 1 -- 172.21.15.37:0/1406148643 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd9a00b9420 con 0x7fd9a0162cf0 2024-09-17T04:33:43.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.650+0000 7fd9a560e640 1 -- 172.21.15.37:0/1406148643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fd9a0050ff0 con 0x7fd9a0162cf0 2024-09-17T04:33:43.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.654+0000 7fd9877fe640 1 -- 172.21.15.37:0/1406148643 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7fd994062d80 con 0x7fd9a0162cf0 2024-09-17T04:33:43.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.656+0000 7fd9a560e640 1 -- 172.21.15.37:0/1406148643 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd98c078b90 msgr2=0x7fd98c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:43.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.656+0000 7fd9a560e640 1 --2- 172.21.15.37:0/1406148643 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd98c078b90 0x7fd98c07b0a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd990000f60 tx=0x7fd990009040 comp rx=0 tx=0).stop 2024-09-17T04:33:43.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.656+0000 7fd9a560e640 1 -- 172.21.15.37:0/1406148643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9a0162cf0 msgr2=0x7fd9a0158130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:43.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.656+0000 7fd9a560e640 1 --2- 172.21.15.37:0/1406148643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9a0162cf0 0x7fd9a0158130 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd99400d6d0 tx=0x7fd99400db90 comp rx=0 tx=0).stop 2024-09-17T04:33:43.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.656+0000 7fd9a560e640 1 -- 172.21.15.37:0/1406148643 shutdown_connections 2024-09-17T04:33:43.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.656+0000 7fd9a560e640 1 --2- 172.21.15.37:0/1406148643 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd98c078b90 0x7fd98c07b0a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.656+0000 7fd9a560e640 1 --2- 172.21.15.37:0/1406148643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9a00b8ad0 0x7fd9a00b8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.656+0000 7fd9a560e640 1 --2- 172.21.15.37:0/1406148643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9a0162cf0 0x7fd9a0158130 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.656+0000 7fd9a560e640 1 --2- 172.21.15.37:0/1406148643 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd9a015e830 0x7fd9a00c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.656+0000 7fd9a560e640 1 -- 172.21.15.37:0/1406148643 >> 172.21.15.37:0/1406148643 conn(0x7fd9a00d4540 msgr2=0x7fd9a01e81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:43.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.656+0000 7fd9a560e640 1 -- 172.21.15.37:0/1406148643 shutdown_connections 2024-09-17T04:33:43.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.656+0000 7fd9a560e640 1 -- 172.21.15.37:0/1406148643 wait complete. 2024-09-17T04:33:43.667 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T04:33:43.668 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:33:43.706 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:33:43.747 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T04:33:43.748 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:33:43.748 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:43.764 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:33:43.765 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:43.821 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:33:43.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.980+0000 7fabc64c5640 1 -- 172.21.15.37:0/1379449355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabc00cd260 msgr2=0x7fabc00b5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:43.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.980+0000 7fabc64c5640 1 --2- 172.21.15.37:0/1379449355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabc00cd260 0x7fabc00b5700 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fabc0073ef0 tx=0x7fabb802aac0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.980+0000 7fabc64c5640 1 -- 172.21.15.37:0/1379449355 shutdown_connections 2024-09-17T04:33:43.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.980+0000 7fabc64c5640 1 --2- 172.21.15.37:0/1379449355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabc00cd260 0x7fabc00b5700 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.980+0000 7fabc64c5640 1 --2- 172.21.15.37:0/1379449355 >> v2:172.21.15.92:3300/0 conn(0x7fabc015c830 0x7fabc00d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.980+0000 7fabc64c5640 1 -- 172.21.15.37:0/1379449355 >> 172.21.15.37:0/1379449355 conn(0x7fabc0071050 msgr2=0x7fabc0071460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:43.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.980+0000 7fabc64c5640 1 -- 172.21.15.37:0/1379449355 shutdown_connections 2024-09-17T04:33:43.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.980+0000 7fabc64c5640 1 -- 172.21.15.37:0/1379449355 wait complete. 2024-09-17T04:33:43.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.982+0000 7fabc64c5640 1 Processor -- start 2024-09-17T04:33:43.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.982+0000 7fabc64c5640 1 -- start start 2024-09-17T04:33:43.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.984+0000 7fabc64c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabc00d1a10 0x7fabc01f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:43.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.984+0000 7fabc64c5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabc015c830 0x7fabc0142040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:43.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.984+0000 7fabc64c5640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fabc01520a0 0x7fabc0142580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:43.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.984+0000 7fabc64c5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fabc00b6240 con 0x7fabc00d1a10 2024-09-17T04:33:43.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.984+0000 7fabc64c5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fabc00b60c0 con 0x7fabc015c830 2024-09-17T04:33:43.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.984+0000 7fabc64c5640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fabc00b63c0 con 0x7fabc01520a0 2024-09-17T04:33:43.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.984+0000 7fabbffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabc00d1a10 0x7fabc01f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:43.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.984+0000 7fabbffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabc00d1a10 0x7fabc01f61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53038/0 (socket says 172.21.15.37:53038) 2024-09-17T04:33:43.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.984+0000 7fabbffff640 1 -- 172.21.15.37:0/1103026168 learned_addr learned my addr 172.21.15.37:0/1103026168 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:43.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.984+0000 7fabbf7fe640 1 --2- 172.21.15.37:0/1103026168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabc015c830 0x7fabc0142040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:43.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.984+0000 7fabc4a62640 1 --2- 172.21.15.37:0/1103026168 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fabc01520a0 0x7fabc0142580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:43.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.984+0000 7fabbffff640 1 -- 172.21.15.37:0/1103026168 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fabc01520a0 msgr2=0x7fabc0142580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:43.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.984+0000 7fabbffff640 1 --2- 172.21.15.37:0/1103026168 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fabc01520a0 0x7fabc0142580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.984+0000 7fabbffff640 1 -- 172.21.15.37:0/1103026168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabc015c830 msgr2=0x7fabc0142040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:43.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.984+0000 7fabbffff640 1 --2- 172.21.15.37:0/1103026168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabc015c830 0x7fabc0142040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:43.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.984+0000 7fabbffff640 1 -- 172.21.15.37:0/1103026168 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc0154440 con 0x7fabc00d1a10 2024-09-17T04:33:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.984+0000 7fabbffff640 1 --2- 172.21.15.37:0/1103026168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabc00d1a10 0x7fabc01f61a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fabb000d740 tx=0x7fabb000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.985+0000 7fabbcff9640 1 -- 172.21.15.37:0/1103026168 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb0017070 con 0x7fabc00d1a10 2024-09-17T04:33:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.985+0000 7fabbcff9640 1 -- 172.21.15.37:0/1103026168 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabb0009d20 con 0x7fabc00d1a10 2024-09-17T04:33:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.985+0000 7fabbcff9640 1 -- 172.21.15.37:0/1103026168 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb00106c0 con 0x7fabc00d1a10 2024-09-17T04:33:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.985+0000 7fabc64c5640 1 -- 172.21.15.37:0/1103026168 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc0154710 con 0x7fabc00d1a10 2024-09-17T04:33:43.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.985+0000 7fabc64c5640 1 -- 172.21.15.37:0/1103026168 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc0154cd0 con 0x7fabc00d1a10 2024-09-17T04:33:43.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.986+0000 7fabc64c5640 1 -- 172.21.15.37:0/1103026168 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc00b6870 con 0x7fabc00d1a10 2024-09-17T04:33:43.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.986+0000 7fabbcff9640 1 -- 172.21.15.37:0/1103026168 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fabb001a070 con 0x7fabc00d1a10 2024-09-17T04:33:43.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.987+0000 7fabbcff9640 1 --2- 172.21.15.37:0/1103026168 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fabac078b90 0x7fabac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:43.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.987+0000 7fabbcff9640 1 -- 172.21.15.37:0/1103026168 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fabb009ae70 con 0x7fabc00d1a10 2024-09-17T04:33:43.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.987+0000 7fabbf7fe640 1 --2- 172.21.15.37:0/1103026168 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fabac078b90 0x7fabac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:43.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.988+0000 7fabbf7fe640 1 --2- 172.21.15.37:0/1103026168 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fabac078b90 0x7fabac07b0a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fabc01566c0 tx=0x7fabb4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:43.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:43.992+0000 7fabbcff9640 1 -- 172.21.15.37:0/1103026168 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fabb0062c50 con 0x7fabc00d1a10 2024-09-17T04:33:44.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.191+0000 7fabc64c5640 1 -- 172.21.15.37:0/1103026168 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fabc01554d0 con 0x7fabc00d1a10 2024-09-17T04:33:44.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.191+0000 7fabbcff9640 1 -- 172.21.15.37:0/1103026168 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fabb0067680 con 0x7fabc00d1a10 2024-09-17T04:33:44.191 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-09-17T04:33:44.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.194+0000 7fabc64c5640 1 -- 172.21.15.37:0/1103026168 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fabac078b90 msgr2=0x7fabac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:44.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.194+0000 7fabc64c5640 1 --2- 172.21.15.37:0/1103026168 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fabac078b90 0x7fabac07b0a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fabc01566c0 tx=0x7fabb4009040 comp rx=0 tx=0).stop 2024-09-17T04:33:44.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.194+0000 7fabc64c5640 1 -- 172.21.15.37:0/1103026168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabc00d1a10 msgr2=0x7fabc01f61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:44.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.194+0000 7fabc64c5640 1 --2- 172.21.15.37:0/1103026168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabc00d1a10 0x7fabc01f61a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fabb000d740 tx=0x7fabb000dc00 comp rx=0 tx=0).stop 2024-09-17T04:33:44.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.194+0000 7fabc64c5640 1 -- 172.21.15.37:0/1103026168 shutdown_connections 2024-09-17T04:33:44.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.194+0000 7fabc64c5640 1 --2- 172.21.15.37:0/1103026168 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fabac078b90 0x7fabac07b0a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.194+0000 7fabc64c5640 1 --2- 172.21.15.37:0/1103026168 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fabc01520a0 0x7fabc0142580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.194+0000 7fabc64c5640 1 --2- 172.21.15.37:0/1103026168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabc015c830 0x7fabc0142040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.194+0000 7fabc64c5640 1 --2- 172.21.15.37:0/1103026168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabc00d1a10 0x7fabc01f61a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.194+0000 7fabc64c5640 1 -- 172.21.15.37:0/1103026168 >> 172.21.15.37:0/1103026168 conn(0x7fabc0071050 msgr2=0x7fabc015b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:44.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.194+0000 7fabc64c5640 1 -- 172.21.15.37:0/1103026168 shutdown_connections 2024-09-17T04:33:44.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.194+0000 7fabc64c5640 1 -- 172.21.15.37:0/1103026168 wait complete. 2024-09-17T04:33:44.205 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:33:44.205 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:44.222 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:33:44.321 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:33:44.323 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:33:44.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.381+0000 7fe41fce6640 1 -- 172.21.15.37:0/381239025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4180b66d0 msgr2=0x7fe4181627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:44.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.381+0000 7fe41fce6640 1 --2- 172.21.15.37:0/381239025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4180b66d0 0x7fe4181627b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe408005fd0 tx=0x7fe40802a5d0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.381+0000 7fe41fce6640 1 -- 172.21.15.37:0/381239025 shutdown_connections 2024-09-17T04:33:44.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.381+0000 7fe41fce6640 1 --2- 172.21.15.37:0/381239025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4180b66d0 0x7fe4181627b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.381+0000 7fe41fce6640 1 -- 172.21.15.37:0/381239025 >> 172.21.15.37:0/381239025 conn(0x7fe4180d4540 msgr2=0x7fe4180d4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:44.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.381+0000 7fe41fce6640 1 -- 172.21.15.37:0/381239025 shutdown_connections 2024-09-17T04:33:44.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.381+0000 7fe41fce6640 1 -- 172.21.15.37:0/381239025 wait complete. 2024-09-17T04:33:44.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.383+0000 7fe41fce6640 1 Processor -- start 2024-09-17T04:33:44.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.383+0000 7fe41fce6640 1 -- start start 2024-09-17T04:33:44.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.384+0000 7fe41fce6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe41815e830 0x7fe4180c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:44.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.384+0000 7fe41fce6640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe418162cf0 0x7fe4181f5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:44.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.384+0000 7fe41fce6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4180cd040 0x7fe4180cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:44.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.384+0000 7fe41fce6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe4180d3230 con 0x7fe41815e830 2024-09-17T04:33:44.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.384+0000 7fe41fce6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe4180d30b0 con 0x7fe4180cd040 2024-09-17T04:33:44.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.384+0000 7fe41fce6640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fe4180d33b0 con 0x7fe418162cf0 2024-09-17T04:33:44.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.384+0000 7fe41da82640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe41815e830 0x7fe4180c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:44.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.384+0000 7fe41da82640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe41815e830 0x7fe4180c9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53052/0 (socket says 172.21.15.37:53052) 2024-09-17T04:33:44.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.384+0000 7fe41da82640 1 -- 172.21.15.37:0/3832857588 learned_addr learned my addr 172.21.15.37:0/3832857588 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:44.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.384+0000 7fe41e283640 1 --2- 172.21.15.37:0/3832857588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4180cd040 0x7fe4180cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:44.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.384+0000 7fe41d281640 1 --2- 172.21.15.37:0/3832857588 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe418162cf0 0x7fe4181f5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:44.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.384+0000 7fe41da82640 1 -- 172.21.15.37:0/3832857588 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe418162cf0 msgr2=0x7fe4181f5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:44.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.384+0000 7fe41da82640 1 --2- 172.21.15.37:0/3832857588 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe418162cf0 0x7fe4181f5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.384+0000 7fe41da82640 1 -- 172.21.15.37:0/3832857588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4180cd040 msgr2=0x7fe4180cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:44.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.384+0000 7fe41da82640 1 --2- 172.21.15.37:0/3832857588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4180cd040 0x7fe4180cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.384+0000 7fe41da82640 1 -- 172.21.15.37:0/3832857588 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4180cacf0 con 0x7fe41815e830 2024-09-17T04:33:44.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.385+0000 7fe41da82640 1 --2- 172.21.15.37:0/3832857588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe41815e830 0x7fe4180c9460 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe41400c3a0 tx=0x7fe41400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:44.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.385+0000 7fe40e7fc640 1 -- 172.21.15.37:0/3832857588 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414017070 con 0x7fe41815e830 2024-09-17T04:33:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.385+0000 7fe40e7fc640 1 -- 172.21.15.37:0/3832857588 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4140053c0 con 0x7fe41815e830 2024-09-17T04:33:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.385+0000 7fe41fce6640 1 -- 172.21.15.37:0/3832857588 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4180cbfa0 con 0x7fe41815e830 2024-09-17T04:33:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.385+0000 7fe40e7fc640 1 -- 172.21.15.37:0/3832857588 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414003850 con 0x7fe41815e830 2024-09-17T04:33:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.385+0000 7fe41fce6640 1 -- 172.21.15.37:0/3832857588 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4180cc560 con 0x7fe41815e830 2024-09-17T04:33:44.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.386+0000 7fe41fce6640 1 -- 172.21.15.37:0/3832857588 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4181627b0 con 0x7fe41815e830 2024-09-17T04:33:44.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.387+0000 7fe40e7fc640 1 -- 172.21.15.37:0/3832857588 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe41401a070 con 0x7fe41815e830 2024-09-17T04:33:44.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.387+0000 7fe40e7fc640 1 --2- 172.21.15.37:0/3832857588 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe3fc078b90 0x7fe3fc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:44.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.387+0000 7fe40e7fc640 1 -- 172.21.15.37:0/3832857588 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe414099fe0 con 0x7fe41815e830 2024-09-17T04:33:44.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.388+0000 7fe41d281640 1 --2- 172.21.15.37:0/3832857588 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe3fc078b90 0x7fe3fc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:44.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.388+0000 7fe41d281640 1 --2- 172.21.15.37:0/3832857588 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe3fc078b90 0x7fe3fc07b0a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe408033000 tx=0x7fe408004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:44.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.392+0000 7fe40e7fc640 1 -- 172.21.15.37:0/3832857588 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe414062540 con 0x7fe41815e830 2024-09-17T04:33:44.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.590+0000 7fe41fce6640 1 -- 172.21.15.37:0/3832857588 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe4180b5fb0 con 0x7fe41815e830 2024-09-17T04:33:44.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.591+0000 7fe40e7fc640 1 -- 172.21.15.37:0/3832857588 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fe414061e40 con 0x7fe41815e830 2024-09-17T04:33:44.591 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-09-17T04:33:44.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.593+0000 7fe41fce6640 1 -- 172.21.15.37:0/3832857588 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe3fc078b90 msgr2=0x7fe3fc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:44.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.593+0000 7fe41fce6640 1 --2- 172.21.15.37:0/3832857588 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe3fc078b90 0x7fe3fc07b0a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe408033000 tx=0x7fe408004730 comp rx=0 tx=0).stop 2024-09-17T04:33:44.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.593+0000 7fe41fce6640 1 -- 172.21.15.37:0/3832857588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe41815e830 msgr2=0x7fe4180c9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:44.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.593+0000 7fe41fce6640 1 --2- 172.21.15.37:0/3832857588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe41815e830 0x7fe4180c9460 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe41400c3a0 tx=0x7fe41400c860 comp rx=0 tx=0).stop 2024-09-17T04:33:44.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.593+0000 7fe41fce6640 1 -- 172.21.15.37:0/3832857588 shutdown_connections 2024-09-17T04:33:44.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.593+0000 7fe41fce6640 1 --2- 172.21.15.37:0/3832857588 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe3fc078b90 0x7fe3fc07b0a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.593+0000 7fe41fce6640 1 --2- 172.21.15.37:0/3832857588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4180cd040 0x7fe4180cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.593+0000 7fe41fce6640 1 --2- 172.21.15.37:0/3832857588 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe418162cf0 0x7fe4181f5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.593+0000 7fe41fce6640 1 --2- 172.21.15.37:0/3832857588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe41815e830 0x7fe4180c9460 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.593+0000 7fe41fce6640 1 -- 172.21.15.37:0/3832857588 >> 172.21.15.37:0/3832857588 conn(0x7fe4180d4540 msgr2=0x7fe41815dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:44.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.593+0000 7fe41fce6640 1 -- 172.21.15.37:0/3832857588 shutdown_connections 2024-09-17T04:33:44.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.593+0000 7fe41fce6640 1 -- 172.21.15.37:0/3832857588 wait complete. 2024-09-17T04:33:44.604 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-09-17T04:33:44.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:44.604 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-17T04:33:44.671 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:33:44.672 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-09-17T04:33:44.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:44.672 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T04:33:44.738 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:33:44.738 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:44.757 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:33:44.757 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:44.814 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:33:44.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.932+0000 7fb674ecc640 1 -- 172.21.15.37:0/525725920 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb6700d0ff0 msgr2=0x7fb6700d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:44.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.932+0000 7fb674ecc640 1 --2- 172.21.15.37:0/525725920 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb6700d0ff0 0x7fb6700d1400 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb668005fd0 tx=0x7fb66802a8b0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.932+0000 7fb674ecc640 1 -- 172.21.15.37:0/525725920 shutdown_connections 2024-09-17T04:33:44.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.932+0000 7fb674ecc640 1 --2- 172.21.15.37:0/525725920 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb6700d0ff0 0x7fb6700d1400 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.932+0000 7fb674ecc640 1 --2- 172.21.15.37:0/525725920 >> v2:172.21.15.92:3300/0 conn(0x7fb670163040 0x7fb6700cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.932+0000 7fb674ecc640 1 -- 172.21.15.37:0/525725920 >> 172.21.15.37:0/525725920 conn(0x7fb67006de00 msgr2=0x7fb67006e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:44.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.932+0000 7fb674ecc640 1 -- 172.21.15.37:0/525725920 shutdown_connections 2024-09-17T04:33:44.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.932+0000 7fb674ecc640 1 -- 172.21.15.37:0/525725920 wait complete. 2024-09-17T04:33:44.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.934+0000 7fb674ecc640 1 Processor -- start 2024-09-17T04:33:44.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.934+0000 7fb674ecc640 1 -- start start 2024-09-17T04:33:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.936+0000 7fb674ecc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6700d0ff0 0x7fb6700b6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.936+0000 7fb674ecc640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb67015fbe0 0x7fb6700c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:44.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.936+0000 7fb674ecc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb670163040 0x7fb6701488a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:44.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.936+0000 7fb674ecc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb6700d32a0 con 0x7fb6700d0ff0 2024-09-17T04:33:44.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.936+0000 7fb674ecc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb6700d3120 con 0x7fb670163040 2024-09-17T04:33:44.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.936+0000 7fb674ecc640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fb6700d3420 con 0x7fb67015fbe0 2024-09-17T04:33:44.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.936+0000 7fb66e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6700d0ff0 0x7fb6700b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:44.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.936+0000 7fb66dd9b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb67015fbe0 0x7fb6700c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:44.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.936+0000 7fb66e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6700d0ff0 0x7fb6700b6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53064/0 (socket says 172.21.15.37:53064) 2024-09-17T04:33:44.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.936+0000 7fb66e59c640 1 -- 172.21.15.37:0/789054996 learned_addr learned my addr 172.21.15.37:0/789054996 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:44.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.936+0000 7fb66ed9d640 1 --2- 172.21.15.37:0/789054996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb670163040 0x7fb6701488a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:44.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.936+0000 7fb66e59c640 1 -- 172.21.15.37:0/789054996 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb67015fbe0 msgr2=0x7fb6700c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:44.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.936+0000 7fb66e59c640 1 --2- 172.21.15.37:0/789054996 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb67015fbe0 0x7fb6700c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.936+0000 7fb66e59c640 1 -- 172.21.15.37:0/789054996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb670163040 msgr2=0x7fb6701488a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:44.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.936+0000 7fb66e59c640 1 --2- 172.21.15.37:0/789054996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb670163040 0x7fb6701488a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:44.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.936+0000 7fb66e59c640 1 -- 172.21.15.37:0/789054996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6700c8100 con 0x7fb6700d0ff0 2024-09-17T04:33:44.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.936+0000 7fb66e59c640 1 --2- 172.21.15.37:0/789054996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6700d0ff0 0x7fb6700b6990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb66000c730 tx=0x7fb66000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:44.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.936+0000 7fb65effd640 1 -- 172.21.15.37:0/789054996 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb660016020 con 0x7fb6700d0ff0 2024-09-17T04:33:44.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.937+0000 7fb65effd640 1 -- 172.21.15.37:0/789054996 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb66000a8f0 con 0x7fb6700d0ff0 2024-09-17T04:33:44.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.937+0000 7fb65effd640 1 -- 172.21.15.37:0/789054996 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66000ad40 con 0x7fb6700d0ff0 2024-09-17T04:33:44.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.937+0000 7fb674ecc640 1 -- 172.21.15.37:0/789054996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6700c83e0 con 0x7fb6700d0ff0 2024-09-17T04:33:44.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.937+0000 7fb674ecc640 1 -- 172.21.15.37:0/789054996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6700c89a0 con 0x7fb6700d0ff0 2024-09-17T04:33:44.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.938+0000 7fb674ecc640 1 -- 172.21.15.37:0/789054996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6700d3940 con 0x7fb6700d0ff0 2024-09-17T04:33:44.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.938+0000 7fb65effd640 1 -- 172.21.15.37:0/789054996 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb660019020 con 0x7fb6700d0ff0 2024-09-17T04:33:44.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.939+0000 7fb65effd640 1 --2- 172.21.15.37:0/789054996 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb650078b90 0x7fb65007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:44.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.939+0000 7fb65effd640 1 -- 172.21.15.37:0/789054996 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb66009ab10 con 0x7fb6700d0ff0 2024-09-17T04:33:44.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.939+0000 7fb66dd9b640 1 --2- 172.21.15.37:0/789054996 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb650078b90 0x7fb65007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:44.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.940+0000 7fb66dd9b640 1 --2- 172.21.15.37:0/789054996 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb650078b90 0x7fb65007b0a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb6700b7850 tx=0x7fb664005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:44.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:44.943+0000 7fb65effd640 1 -- 172.21.15.37:0/789054996 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb660020020 con 0x7fb6700d0ff0 2024-09-17T04:33:45.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.143+0000 7fb674ecc640 1 -- 172.21.15.37:0/789054996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb67015e890 con 0x7fb6700d0ff0 2024-09-17T04:33:45.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.143+0000 7fb65effd640 1 -- 172.21.15.37:0/789054996 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fb6700d1940 con 0x7fb6700d0ff0 2024-09-17T04:33:45.143 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-09-17T04:33:45.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.145+0000 7fb674ecc640 1 -- 172.21.15.37:0/789054996 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb650078b90 msgr2=0x7fb65007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:45.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.145+0000 7fb674ecc640 1 --2- 172.21.15.37:0/789054996 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb650078b90 0x7fb65007b0a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb6700b7850 tx=0x7fb664005f90 comp rx=0 tx=0).stop 2024-09-17T04:33:45.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.145+0000 7fb674ecc640 1 -- 172.21.15.37:0/789054996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6700d0ff0 msgr2=0x7fb6700b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:45.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.145+0000 7fb674ecc640 1 --2- 172.21.15.37:0/789054996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6700d0ff0 0x7fb6700b6990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb66000c730 tx=0x7fb66000cbf0 comp rx=0 tx=0).stop 2024-09-17T04:33:45.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.145+0000 7fb674ecc640 1 -- 172.21.15.37:0/789054996 shutdown_connections 2024-09-17T04:33:45.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.145+0000 7fb674ecc640 1 --2- 172.21.15.37:0/789054996 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb650078b90 0x7fb65007b0a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:45.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.145+0000 7fb674ecc640 1 --2- 172.21.15.37:0/789054996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb670163040 0x7fb6701488a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:45.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.145+0000 7fb674ecc640 1 --2- 172.21.15.37:0/789054996 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb67015fbe0 0x7fb6700c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:45.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.145+0000 7fb674ecc640 1 --2- 172.21.15.37:0/789054996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6700d0ff0 0x7fb6700b6990 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:45.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.145+0000 7fb674ecc640 1 -- 172.21.15.37:0/789054996 >> 172.21.15.37:0/789054996 conn(0x7fb67006de00 msgr2=0x7fb67015dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:45.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.145+0000 7fb674ecc640 1 -- 172.21.15.37:0/789054996 shutdown_connections 2024-09-17T04:33:45.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.145+0000 7fb674ecc640 1 -- 172.21.15.37:0/789054996 wait complete. 2024-09-17T04:33:45.156 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:33:45.156 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:45.175 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:33:45.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.293+0000 7ff7b2ee5640 1 -- 172.21.15.37:0/2931941042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ac15c4e0 msgr2=0x7ff7ac0cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:45.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.293+0000 7ff7b2ee5640 1 --2- 172.21.15.37:0/2931941042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ac15c4e0 0x7ff7ac0cfb80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0ce9c0 tx=0x7ff79402a560 comp rx=0 tx=0).stop 2024-09-17T04:33:45.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.293+0000 7ff7b2ee5640 1 -- 172.21.15.37:0/2931941042 shutdown_connections 2024-09-17T04:33:45.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.293+0000 7ff7b2ee5640 1 --2- 172.21.15.37:0/2931941042 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff7ac15ed10 0x7ff7ac15f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:45.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.293+0000 7ff7b2ee5640 1 --2- 172.21.15.37:0/2931941042 >> v2:172.21.15.92:3300/0 conn(0x7ff7ac0b5f20 0x7ff7ac0b6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:45.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.293+0000 7ff7b2ee5640 1 --2- 172.21.15.37:0/2931941042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ac15c4e0 0x7ff7ac0cfb80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:45.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.293+0000 7ff7b2ee5640 1 -- 172.21.15.37:0/2931941042 >> 172.21.15.37:0/2931941042 conn(0x7ff7ac0d44a0 msgr2=0x7ff7ac0d48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:45.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.293+0000 7ff7b2ee5640 1 -- 172.21.15.37:0/2931941042 shutdown_connections 2024-09-17T04:33:45.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.293+0000 7ff7b2ee5640 1 -- 172.21.15.37:0/2931941042 wait complete. 2024-09-17T04:33:45.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.295+0000 7ff7b2ee5640 1 Processor -- start 2024-09-17T04:33:45.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.295+0000 7ff7b2ee5640 1 -- start start 2024-09-17T04:33:45.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.297+0000 7ff7b2ee5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ac0b5f20 0x7ff7ac1f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:45.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.297+0000 7ff7b2ee5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7ac15c4e0 0x7ff7ac1f8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:45.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.297+0000 7ff7b2ee5640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff7ac15ed10 0x7ff7ac161260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:45.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.297+0000 7ff7b2ee5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac0d2ff0 con 0x7ff7ac0b5f20 2024-09-17T04:33:45.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.297+0000 7ff7b2ee5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac0d2e70 con 0x7ff7ac15c4e0 2024-09-17T04:33:45.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.297+0000 7ff7b2ee5640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7ff7ac0d3170 con 0x7ff7ac15ed10 2024-09-17T04:33:45.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.297+0000 7ff7b0c81640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ac0b5f20 0x7ff7ac1f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:45.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.297+0000 7ff7b0c81640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ac0b5f20 0x7ff7ac1f6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49200/0 (socket says 172.21.15.37:49200) 2024-09-17T04:33:45.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.297+0000 7ff7b0c81640 1 -- 172.21.15.37:0/924353162 learned_addr learned my addr 172.21.15.37:0/924353162 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:45.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.297+0000 7ff7b1482640 1 --2- 172.21.15.37:0/924353162 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff7ac15ed10 0x7ff7ac161260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:45.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.297+0000 7ff7abfff640 1 --2- 172.21.15.37:0/924353162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7ac15c4e0 0x7ff7ac1f8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:45.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.297+0000 7ff7b0c81640 1 -- 172.21.15.37:0/924353162 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff7ac15ed10 msgr2=0x7ff7ac161260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:45.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.297+0000 7ff7b0c81640 1 --2- 172.21.15.37:0/924353162 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff7ac15ed10 0x7ff7ac161260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:45.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.297+0000 7ff7b0c81640 1 -- 172.21.15.37:0/924353162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7ac15c4e0 msgr2=0x7ff7ac1f8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:45.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.297+0000 7ff7b0c81640 1 --2- 172.21.15.37:0/924353162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7ac15c4e0 0x7ff7ac1f8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:45.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.297+0000 7ff7b0c81640 1 -- 172.21.15.37:0/924353162 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7ac1f56c0 con 0x7ff7ac0b5f20 2024-09-17T04:33:45.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.297+0000 7ff7b0c81640 1 --2- 172.21.15.37:0/924353162 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ac0b5f20 0x7ff7ac1f6140 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff794034040 tx=0x7ff7940039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:45.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.297+0000 7ff7a97fa640 1 -- 172.21.15.37:0/924353162 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79403e030 con 0x7ff7ac0b5f20 2024-09-17T04:33:45.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.298+0000 7ff7a97fa640 1 -- 172.21.15.37:0/924353162 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7940047a0 con 0x7ff7ac0b5f20 2024-09-17T04:33:45.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.298+0000 7ff7b2ee5640 1 -- 172.21.15.37:0/924353162 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7ac1f9750 con 0x7ff7ac0b5f20 2024-09-17T04:33:45.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.298+0000 7ff7a97fa640 1 -- 172.21.15.37:0/924353162 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff794035040 con 0x7ff7ac0b5f20 2024-09-17T04:33:45.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.298+0000 7ff7b2ee5640 1 -- 172.21.15.37:0/924353162 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7ac1f5940 con 0x7ff7ac0b5f20 2024-09-17T04:33:45.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.299+0000 7ff7b2ee5640 1 -- 172.21.15.37:0/924353162 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7ac0d4950 con 0x7ff7ac0b5f20 2024-09-17T04:33:45.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.299+0000 7ff7a97fa640 1 -- 172.21.15.37:0/924353162 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff794041030 con 0x7ff7ac0b5f20 2024-09-17T04:33:45.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.300+0000 7ff7a97fa640 1 --2- 172.21.15.37:0/924353162 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff78c078b90 0x7ff78c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:45.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.300+0000 7ff7a97fa640 1 -- 172.21.15.37:0/924353162 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff7940bb650 con 0x7ff7ac0b5f20 2024-09-17T04:33:45.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.300+0000 7ff7abfff640 1 --2- 172.21.15.37:0/924353162 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff78c078b90 0x7ff78c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:45.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.301+0000 7ff7abfff640 1 --2- 172.21.15.37:0/924353162 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff78c078b90 0x7ff78c07b0a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00a490 tx=0x7ff79c00a420 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:45.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.305+0000 7ff7a97fa640 1 -- 172.21.15.37:0/924353162 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff7ac1f56c0 con 0x7ff7ac0b5f20 2024-09-17T04:33:45.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.504+0000 7ff7b2ee5640 1 -- 172.21.15.37:0/924353162 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff7ac161fc0 con 0x7ff7ac0b5f20 2024-09-17T04:33:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.505+0000 7ff7a97fa640 1 -- 172.21.15.37:0/924353162 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7ff7940833a0 con 0x7ff7ac0b5f20 2024-09-17T04:33:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-09-17T04:33:45.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.507+0000 7ff7b2ee5640 1 -- 172.21.15.37:0/924353162 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff78c078b90 msgr2=0x7ff78c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:45.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.507+0000 7ff7b2ee5640 1 --2- 172.21.15.37:0/924353162 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff78c078b90 0x7ff78c07b0a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00a490 tx=0x7ff79c00a420 comp rx=0 tx=0).stop 2024-09-17T04:33:45.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.507+0000 7ff7b2ee5640 1 -- 172.21.15.37:0/924353162 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ac0b5f20 msgr2=0x7ff7ac1f6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:45.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.507+0000 7ff7b2ee5640 1 --2- 172.21.15.37:0/924353162 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ac0b5f20 0x7ff7ac1f6140 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff794034040 tx=0x7ff7940039e0 comp rx=0 tx=0).stop 2024-09-17T04:33:45.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.507+0000 7ff7b2ee5640 1 -- 172.21.15.37:0/924353162 shutdown_connections 2024-09-17T04:33:45.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.507+0000 7ff7b2ee5640 1 --2- 172.21.15.37:0/924353162 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff78c078b90 0x7ff78c07b0a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:45.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.507+0000 7ff7b2ee5640 1 --2- 172.21.15.37:0/924353162 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff7ac15ed10 0x7ff7ac161260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:45.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.507+0000 7ff7b2ee5640 1 --2- 172.21.15.37:0/924353162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7ac15c4e0 0x7ff7ac1f8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:45.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.507+0000 7ff7b2ee5640 1 --2- 172.21.15.37:0/924353162 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ac0b5f20 0x7ff7ac1f6140 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:45.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.507+0000 7ff7b2ee5640 1 -- 172.21.15.37:0/924353162 >> 172.21.15.37:0/924353162 conn(0x7ff7ac0d44a0 msgr2=0x7ff7ac0d18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:45.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.507+0000 7ff7b2ee5640 1 -- 172.21.15.37:0/924353162 shutdown_connections 2024-09-17T04:33:45.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.507+0000 7ff7b2ee5640 1 -- 172.21.15.37:0/924353162 wait complete. 2024-09-17T04:33:45.518 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi092.front.sepia.ceph.com... 2024-09-17T04:33:45.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:45.519 DEBUG:teuthology.orchestra.run.smithi092:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-17T04:33:45.549 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:33:45.549 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2024-09-17T04:33:45.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:45.549 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T04:33:45.616 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:33:45.616 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:45.633 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:33:45.633 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:45.690 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:33:45.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.847+0000 7faab1c69640 1 -- 172.21.15.37:0/2719035462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaac15d960 msgr2=0x7faaac0cf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:45.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.847+0000 7faab1c69640 1 --2- 172.21.15.37:0/2719035462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaac15d960 0x7faaac0cf1e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7faaac0b4340 tx=0x7faa9c02a560 comp rx=0 tx=0).stop 2024-09-17T04:33:45.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.848+0000 7faab1c69640 1 -- 172.21.15.37:0/2719035462 shutdown_connections 2024-09-17T04:33:45.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.848+0000 7faab1c69640 1 --2- 172.21.15.37:0/2719035462 >> v2:172.21.15.92:3300/0 conn(0x7faaac15c740 0x7faaac160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:45.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.848+0000 7faab1c69640 1 --2- 172.21.15.37:0/2719035462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaac15d960 0x7faaac0cf1e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:45.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.848+0000 7faab1c69640 1 -- 172.21.15.37:0/2719035462 >> 172.21.15.37:0/2719035462 conn(0x7faaac069120 msgr2=0x7faaac069530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:45.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.848+0000 7faab1c69640 1 -- 172.21.15.37:0/2719035462 shutdown_connections 2024-09-17T04:33:45.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.848+0000 7faab1c69640 1 -- 172.21.15.37:0/2719035462 wait complete. 2024-09-17T04:33:45.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.849+0000 7faab1c69640 1 Processor -- start 2024-09-17T04:33:45.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.850+0000 7faab1c69640 1 -- start start 2024-09-17T04:33:45.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.851+0000 7faab1c69640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7faaac0cf720 0x7faaac14e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:45.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.851+0000 7faab1c69640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaac15c740 0x7faaac15e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:45.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.851+0000 7faab1c69640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaac15d960 0x7faaac15ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:45.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.851+0000 7faab1c69640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faaac1573f0 con 0x7faaac15c740 2024-09-17T04:33:45.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.851+0000 7faab1c69640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faaac157270 con 0x7faaac15d960 2024-09-17T04:33:45.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.851+0000 7faab1c69640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7faaac0cd460 con 0x7faaac0cf720 2024-09-17T04:33:45.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.851+0000 7faaaaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaac15c740 0x7faaac15e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:45.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.851+0000 7faaaaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaac15c740 0x7faaac15e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49214/0 (socket says 172.21.15.37:49214) 2024-09-17T04:33:45.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.851+0000 7faaaaffd640 1 -- 172.21.15.37:0/2174482859 learned_addr learned my addr 172.21.15.37:0/2174482859 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:45.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.851+0000 7faaab7fe640 1 --2- 172.21.15.37:0/2174482859 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7faaac0cf720 0x7faaac14e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:45.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.851+0000 7faaabfff640 1 --2- 172.21.15.37:0/2174482859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaac15d960 0x7faaac15ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:45.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.851+0000 7faaaaffd640 1 -- 172.21.15.37:0/2174482859 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7faaac0cf720 msgr2=0x7faaac14e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:45.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.851+0000 7faaaaffd640 1 --2- 172.21.15.37:0/2174482859 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7faaac0cf720 0x7faaac14e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:45.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.851+0000 7faaaaffd640 1 -- 172.21.15.37:0/2174482859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaac15d960 msgr2=0x7faaac15ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:45.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.851+0000 7faaaaffd640 1 --2- 172.21.15.37:0/2174482859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaac15d960 0x7faaac15ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:45.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.851+0000 7faaaaffd640 1 -- 172.21.15.37:0/2174482859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaac14ed90 con 0x7faaac15c740 2024-09-17T04:33:45.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.852+0000 7faaaaffd640 1 --2- 172.21.15.37:0/2174482859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaac15c740 0x7faaac15e7b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7faaa000c3a0 tx=0x7faaa000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:45.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.852+0000 7faa93fff640 1 -- 172.21.15.37:0/2174482859 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa0017070 con 0x7faaac15c740 2024-09-17T04:33:45.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.852+0000 7faa93fff640 1 -- 172.21.15.37:0/2174482859 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faaa00053c0 con 0x7faaac15c740 2024-09-17T04:33:45.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.852+0000 7faab1c69640 1 -- 172.21.15.37:0/2174482859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaac0ca160 con 0x7faaac15c740 2024-09-17T04:33:45.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.852+0000 7faa93fff640 1 -- 172.21.15.37:0/2174482859 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa0003850 con 0x7faaac15c740 2024-09-17T04:33:45.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.852+0000 7faab1c69640 1 -- 172.21.15.37:0/2174482859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaac0cc570 con 0x7faaac15c740 2024-09-17T04:33:45.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.853+0000 7faab1c69640 1 -- 172.21.15.37:0/2174482859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaac0cf1e0 con 0x7faaac15c740 2024-09-17T04:33:45.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.854+0000 7faa93fff640 1 -- 172.21.15.37:0/2174482859 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faaa001a070 con 0x7faaac15c740 2024-09-17T04:33:45.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.854+0000 7faa93fff640 1 --2- 172.21.15.37:0/2174482859 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7faa98078bc0 0x7faa9807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:45.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.854+0000 7faa93fff640 1 -- 172.21.15.37:0/2174482859 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7faaa0099f50 con 0x7faaac15c740 2024-09-17T04:33:45.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.854+0000 7faaab7fe640 1 --2- 172.21.15.37:0/2174482859 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7faa98078bc0 0x7faa9807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:45.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.855+0000 7faaab7fe640 1 --2- 172.21.15.37:0/2174482859 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7faa98078bc0 0x7faa9807b0d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7faa9c033000 tx=0x7faa9c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:45.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:45.859+0000 7faa93fff640 1 -- 172.21.15.37:0/2174482859 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7faaac14ed90 con 0x7faaac15c740 2024-09-17T04:33:46.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.055+0000 7faab1c69640 1 -- 172.21.15.37:0/2174482859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7faaac0c57d0 con 0x7faaac15c740 2024-09-17T04:33:46.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.055+0000 7faa93fff640 1 -- 172.21.15.37:0/2174482859 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7faaac0cfb30 con 0x7faaac15c740 2024-09-17T04:33:46.055 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-09-17T04:33:46.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.057+0000 7faab1c69640 1 -- 172.21.15.37:0/2174482859 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7faa98078bc0 msgr2=0x7faa9807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:46.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.057+0000 7faab1c69640 1 --2- 172.21.15.37:0/2174482859 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7faa98078bc0 0x7faa9807b0d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7faa9c033000 tx=0x7faa9c004730 comp rx=0 tx=0).stop 2024-09-17T04:33:46.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.058+0000 7faab1c69640 1 -- 172.21.15.37:0/2174482859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaac15c740 msgr2=0x7faaac15e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:46.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.058+0000 7faab1c69640 1 --2- 172.21.15.37:0/2174482859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaac15c740 0x7faaac15e7b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7faaa000c3a0 tx=0x7faaa000c860 comp rx=0 tx=0).stop 2024-09-17T04:33:46.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.058+0000 7faab1c69640 1 -- 172.21.15.37:0/2174482859 shutdown_connections 2024-09-17T04:33:46.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.058+0000 7faab1c69640 1 --2- 172.21.15.37:0/2174482859 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7faa98078bc0 0x7faa9807b0d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.058+0000 7faab1c69640 1 --2- 172.21.15.37:0/2174482859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaac15d960 0x7faaac15ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.058+0000 7faab1c69640 1 --2- 172.21.15.37:0/2174482859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaac15c740 0x7faaac15e7b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.058+0000 7faab1c69640 1 --2- 172.21.15.37:0/2174482859 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7faaac0cf720 0x7faaac14e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.058+0000 7faab1c69640 1 -- 172.21.15.37:0/2174482859 >> 172.21.15.37:0/2174482859 conn(0x7faaac069120 msgr2=0x7faaac0d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:46.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.058+0000 7faab1c69640 1 -- 172.21.15.37:0/2174482859 shutdown_connections 2024-09-17T04:33:46.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.058+0000 7faab1c69640 1 -- 172.21.15.37:0/2174482859 wait complete. 2024-09-17T04:33:46.069 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:33:46.069 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:46.085 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:33:46.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.243+0000 7f45b3bad640 1 -- 172.21.15.37:0/48615644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45ac0d15c0 msgr2=0x7f45ac160050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:46.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.243+0000 7f45b3bad640 1 --2- 172.21.15.37:0/48615644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45ac0d15c0 0x7f45ac160050 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f459c009830 tx=0x7f459c02a480 comp rx=0 tx=0).stop 2024-09-17T04:33:46.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.243+0000 7f45b3bad640 1 -- 172.21.15.37:0/48615644 shutdown_connections 2024-09-17T04:33:46.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.243+0000 7f45b3bad640 1 --2- 172.21.15.37:0/48615644 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f45ac160590 0x7f45ac0cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.243+0000 7f45b3bad640 1 --2- 172.21.15.37:0/48615644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45ac0d15c0 0x7f45ac160050 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.243+0000 7f45b3bad640 1 -- 172.21.15.37:0/48615644 >> 172.21.15.37:0/48615644 conn(0x7f45ac06dde0 msgr2=0x7f45ac06e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:46.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.243+0000 7f45b3bad640 1 -- 172.21.15.37:0/48615644 shutdown_connections 2024-09-17T04:33:46.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.243+0000 7f45b3bad640 1 -- 172.21.15.37:0/48615644 wait complete. 2024-09-17T04:33:46.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.245+0000 7f45b3bad640 1 Processor -- start 2024-09-17T04:33:46.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.245+0000 7f45b3bad640 1 -- start start 2024-09-17T04:33:46.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.246+0000 7f45b3bad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45ac0d11b0 0x7f45ac152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:46.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.246+0000 7f45b3bad640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45ac0d15c0 0x7f45ac15bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:46.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.246+0000 7f45b3bad640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f45ac160590 0x7f45ac0b9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:46.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.246+0000 7f45b3bad640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac0cfba0 con 0x7f45ac0d11b0 2024-09-17T04:33:46.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.246+0000 7f45b3bad640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac0cfa20 con 0x7f45ac0d15c0 2024-09-17T04:33:46.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.246+0000 7f45b3bad640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f45ac162b70 con 0x7f45ac160590 2024-09-17T04:33:46.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.246+0000 7f45b1949640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45ac0d11b0 0x7f45ac152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:46.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.246+0000 7f45b1949640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45ac0d11b0 0x7f45ac152f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49232/0 (socket says 172.21.15.37:49232) 2024-09-17T04:33:46.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.246+0000 7f45b1949640 1 -- 172.21.15.37:0/3324070909 learned_addr learned my addr 172.21.15.37:0/3324070909 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:46.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.247+0000 7f45b214a640 1 --2- 172.21.15.37:0/3324070909 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f45ac160590 0x7f45ac0b9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:46.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.247+0000 7f45b1148640 1 --2- 172.21.15.37:0/3324070909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45ac0d15c0 0x7f45ac15bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:46.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.247+0000 7f45b1949640 1 -- 172.21.15.37:0/3324070909 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f45ac160590 msgr2=0x7f45ac0b9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:46.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.247+0000 7f45b1949640 1 --2- 172.21.15.37:0/3324070909 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f45ac160590 0x7f45ac0b9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.247+0000 7f45b1949640 1 -- 172.21.15.37:0/3324070909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45ac0d15c0 msgr2=0x7f45ac15bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:46.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.247+0000 7f45b1949640 1 --2- 172.21.15.37:0/3324070909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45ac0d15c0 0x7f45ac15bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.247+0000 7f45b1949640 1 -- 172.21.15.37:0/3324070909 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45ac1596f0 con 0x7f45ac0d11b0 2024-09-17T04:33:46.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.247+0000 7f45b1148640 1 --2- 172.21.15.37:0/3324070909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45ac0d15c0 0x7f45ac15bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:46.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.247+0000 7f45b1949640 1 --2- 172.21.15.37:0/3324070909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45ac0d11b0 0x7f45ac152f00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f45a800c3f0 tx=0x7f45a800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:46.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.247+0000 7f45a27fc640 1 -- 172.21.15.37:0/3324070909 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8017070 con 0x7f45ac0d11b0 2024-09-17T04:33:46.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.247+0000 7f45a27fc640 1 -- 172.21.15.37:0/3324070909 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45a80035f0 con 0x7f45ac0d11b0 2024-09-17T04:33:46.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.247+0000 7f45b3bad640 1 -- 172.21.15.37:0/3324070909 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45ac1599d0 con 0x7f45ac0d11b0 2024-09-17T04:33:46.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.247+0000 7f45a27fc640 1 -- 172.21.15.37:0/3324070909 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8003a40 con 0x7f45ac0d11b0 2024-09-17T04:33:46.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.247+0000 7f45b3bad640 1 -- 172.21.15.37:0/3324070909 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45ac159f90 con 0x7f45ac0d11b0 2024-09-17T04:33:46.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.248+0000 7f45b3bad640 1 -- 172.21.15.37:0/3324070909 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45ac0b7c10 con 0x7f45ac0d11b0 2024-09-17T04:33:46.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.249+0000 7f45a27fc640 1 -- 172.21.15.37:0/3324070909 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f45a801a070 con 0x7f45ac0d11b0 2024-09-17T04:33:46.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.249+0000 7f45a27fc640 1 --2- 172.21.15.37:0/3324070909 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4590078bc0 0x7f459007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:46.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.249+0000 7f45a27fc640 1 -- 172.21.15.37:0/3324070909 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f45a809a120 con 0x7f45ac0d11b0 2024-09-17T04:33:46.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.250+0000 7f45b1148640 1 --2- 172.21.15.37:0/3324070909 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4590078bc0 0x7f459007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:46.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.250+0000 7f45b1148640 1 --2- 172.21.15.37:0/3324070909 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4590078bc0 0x7f459007b0d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f459c034000 tx=0x7f459c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:46.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.254+0000 7f45a27fc640 1 -- 172.21.15.37:0/3324070909 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f45ac0b7c10 con 0x7f45ac0d11b0 2024-09-17T04:33:46.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.450+0000 7f45b3bad640 1 -- 172.21.15.37:0/3324070909 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f45ac160050 con 0x7f45ac0d11b0 2024-09-17T04:33:46.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.451+0000 7f45a27fc640 1 -- 172.21.15.37:0/3324070909 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f45a80624d0 con 0x7f45ac0d11b0 2024-09-17T04:33:46.451 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-09-17T04:33:46.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.453+0000 7f45b3bad640 1 -- 172.21.15.37:0/3324070909 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4590078bc0 msgr2=0x7f459007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:46.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.453+0000 7f45b3bad640 1 --2- 172.21.15.37:0/3324070909 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4590078bc0 0x7f459007b0d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f459c034000 tx=0x7f459c004730 comp rx=0 tx=0).stop 2024-09-17T04:33:46.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.453+0000 7f45b3bad640 1 -- 172.21.15.37:0/3324070909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45ac0d11b0 msgr2=0x7f45ac152f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:46.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.453+0000 7f45b3bad640 1 --2- 172.21.15.37:0/3324070909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45ac0d11b0 0x7f45ac152f00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f45a800c3f0 tx=0x7f45a800c8b0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.453+0000 7f45b3bad640 1 -- 172.21.15.37:0/3324070909 shutdown_connections 2024-09-17T04:33:46.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.453+0000 7f45b3bad640 1 --2- 172.21.15.37:0/3324070909 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4590078bc0 0x7f459007b0d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.453+0000 7f45b3bad640 1 --2- 172.21.15.37:0/3324070909 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f45ac160590 0x7f45ac0b9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.453+0000 7f45b3bad640 1 --2- 172.21.15.37:0/3324070909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45ac0d15c0 0x7f45ac15bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.453+0000 7f45b3bad640 1 --2- 172.21.15.37:0/3324070909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45ac0d11b0 0x7f45ac152f00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.453+0000 7f45b3bad640 1 -- 172.21.15.37:0/3324070909 >> 172.21.15.37:0/3324070909 conn(0x7f45ac06dde0 msgr2=0x7f45ac15d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:46.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.453+0000 7f45b3bad640 1 -- 172.21.15.37:0/3324070909 shutdown_connections 2024-09-17T04:33:46.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.453+0000 7f45b3bad640 1 -- 172.21.15.37:0/3324070909 wait complete. 2024-09-17T04:33:46.464 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-09-17T04:33:46.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:46.464 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-09-17T04:33:46.490 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:33:46.490 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-09-17T04:33:46.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:46.490 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-17T04:33:46.547 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:33:46.547 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:46.566 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:33:46.566 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:46.625 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:33:46.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.742+0000 7f315ebf1640 1 -- 172.21.15.37:0/2486023040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31580cd090 msgr2=0x7f31580727f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:46.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.742+0000 7f315ebf1640 1 --2- 172.21.15.37:0/2486023040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31580cd090 0x7f31580727f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3158073ef0 tx=0x7f314802aa30 comp rx=0 tx=0).stop 2024-09-17T04:33:46.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.742+0000 7f315ebf1640 1 -- 172.21.15.37:0/2486023040 shutdown_connections 2024-09-17T04:33:46.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.742+0000 7f315ebf1640 1 --2- 172.21.15.37:0/2486023040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31580cd090 0x7f31580727f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.742+0000 7f315ebf1640 1 --2- 172.21.15.37:0/2486023040 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f31580d1a10 0x7f31580ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.742+0000 7f315ebf1640 1 --2- 172.21.15.37:0/2486023040 >> v2:172.21.15.92:3300/0 conn(0x7f315815c830 0x7f31580d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.742+0000 7f315ebf1640 1 -- 172.21.15.37:0/2486023040 >> 172.21.15.37:0/2486023040 conn(0x7f3158071050 msgr2=0x7f3158071460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:46.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.742+0000 7f315ebf1640 1 -- 172.21.15.37:0/2486023040 shutdown_connections 2024-09-17T04:33:46.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.742+0000 7f315ebf1640 1 -- 172.21.15.37:0/2486023040 wait complete. 2024-09-17T04:33:46.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.744+0000 7f315ebf1640 1 Processor -- start 2024-09-17T04:33:46.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.744+0000 7f315ebf1640 1 -- start start 2024-09-17T04:33:46.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.746+0000 7f315ebf1640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f31580cd090 0x7f315814e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:46.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.746+0000 7f315ebf1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31580d1a10 0x7f3158155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:46.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.746+0000 7f315ebf1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f315815c830 0x7f3158156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:46.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.746+0000 7f315ebf1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f31580730c0 con 0x7f31580d1a10 2024-09-17T04:33:46.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.746+0000 7f315ebf1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3158072f40 con 0x7f315815c830 2024-09-17T04:33:46.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.746+0000 7f315ebf1640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f3158073240 con 0x7f31580cd090 2024-09-17T04:33:46.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.746+0000 7f3157fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31580d1a10 0x7f3158155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:46.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.746+0000 7f3157fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31580d1a10 0x7f3158155b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49238/0 (socket says 172.21.15.37:49238) 2024-09-17T04:33:46.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.746+0000 7f3157fff640 1 -- 172.21.15.37:0/1218531088 learned_addr learned my addr 172.21.15.37:0/1218531088 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:46.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.746+0000 7f315c98d640 1 --2- 172.21.15.37:0/1218531088 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f31580cd090 0x7f315814e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:46.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.746+0000 7f315d18e640 1 --2- 172.21.15.37:0/1218531088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f315815c830 0x7f3158156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:46.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.746+0000 7f3157fff640 1 -- 172.21.15.37:0/1218531088 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f31580cd090 msgr2=0x7f315814e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:46.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.746+0000 7f3157fff640 1 --2- 172.21.15.37:0/1218531088 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f31580cd090 0x7f315814e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.746+0000 7f3157fff640 1 -- 172.21.15.37:0/1218531088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f315815c830 msgr2=0x7f3158156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:46.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.746+0000 7f3157fff640 1 --2- 172.21.15.37:0/1218531088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f315815c830 0x7f3158156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.746+0000 7f3157fff640 1 -- 172.21.15.37:0/1218531088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3158154130 con 0x7f31580d1a10 2024-09-17T04:33:46.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.746+0000 7f3157fff640 1 --2- 172.21.15.37:0/1218531088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31580d1a10 0x7f3158155b20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f314000d650 tx=0x7f314000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:46.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.747+0000 7f31557fa640 1 -- 172.21.15.37:0/1218531088 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140017070 con 0x7f31580d1a10 2024-09-17T04:33:46.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.747+0000 7f31557fa640 1 -- 172.21.15.37:0/1218531088 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3140004310 con 0x7f31580d1a10 2024-09-17T04:33:46.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.747+0000 7f31557fa640 1 -- 172.21.15.37:0/1218531088 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140005280 con 0x7f31580d1a10 2024-09-17T04:33:46.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.747+0000 7f315ebf1640 1 -- 172.21.15.37:0/1218531088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3158154410 con 0x7f31580d1a10 2024-09-17T04:33:46.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.747+0000 7f315ebf1640 1 -- 172.21.15.37:0/1218531088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3158153860 con 0x7f31580d1a10 2024-09-17T04:33:46.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.748+0000 7f315ebf1640 1 -- 172.21.15.37:0/1218531088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3158159e90 con 0x7f31580d1a10 2024-09-17T04:33:46.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.748+0000 7f31557fa640 1 -- 172.21.15.37:0/1218531088 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3140005420 con 0x7f31580d1a10 2024-09-17T04:33:46.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.749+0000 7f31557fa640 1 --2- 172.21.15.37:0/1218531088 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3138078bc0 0x7f313807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:46.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.749+0000 7f31557fa640 1 -- 172.21.15.37:0/1218531088 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f314009b0d0 con 0x7f31580d1a10 2024-09-17T04:33:46.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.749+0000 7f315c98d640 1 --2- 172.21.15.37:0/1218531088 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3138078bc0 0x7f313807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:46.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.750+0000 7f315c98d640 1 --2- 172.21.15.37:0/1218531088 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3138078bc0 0x7f313807b0d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3150005fd0 tx=0x7f3150005c50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:46.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.753+0000 7f31557fa640 1 -- 172.21.15.37:0/1218531088 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3140020020 con 0x7f31580d1a10 2024-09-17T04:33:46.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.949+0000 7f315ebf1640 1 -- 172.21.15.37:0/1218531088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f31580ccb50 con 0x7f31580d1a10 2024-09-17T04:33:46.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.949+0000 7f31557fa640 1 -- 172.21.15.37:0/1218531088 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f3140063510 con 0x7f31580d1a10 2024-09-17T04:33:46.949 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-09-17T04:33:46.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.952+0000 7f315ebf1640 1 -- 172.21.15.37:0/1218531088 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3138078bc0 msgr2=0x7f313807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:46.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.952+0000 7f315ebf1640 1 --2- 172.21.15.37:0/1218531088 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3138078bc0 0x7f313807b0d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3150005fd0 tx=0x7f3150005c50 comp rx=0 tx=0).stop 2024-09-17T04:33:46.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.952+0000 7f315ebf1640 1 -- 172.21.15.37:0/1218531088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31580d1a10 msgr2=0x7f3158155b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:46.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.952+0000 7f315ebf1640 1 --2- 172.21.15.37:0/1218531088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31580d1a10 0x7f3158155b20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f314000d650 tx=0x7f314000db10 comp rx=0 tx=0).stop 2024-09-17T04:33:46.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.952+0000 7f315ebf1640 1 -- 172.21.15.37:0/1218531088 shutdown_connections 2024-09-17T04:33:46.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.952+0000 7f315ebf1640 1 --2- 172.21.15.37:0/1218531088 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3138078bc0 0x7f313807b0d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.952+0000 7f315ebf1640 1 --2- 172.21.15.37:0/1218531088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f315815c830 0x7f3158156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.952+0000 7f315ebf1640 1 --2- 172.21.15.37:0/1218531088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31580d1a10 0x7f3158155b20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.952+0000 7f315ebf1640 1 --2- 172.21.15.37:0/1218531088 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f31580cd090 0x7f315814e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:46.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.952+0000 7f315ebf1640 1 -- 172.21.15.37:0/1218531088 >> 172.21.15.37:0/1218531088 conn(0x7f3158071050 msgr2=0x7f31580b7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:46.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.952+0000 7f315ebf1640 1 -- 172.21.15.37:0/1218531088 shutdown_connections 2024-09-17T04:33:46.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:46.952+0000 7f315ebf1640 1 -- 172.21.15.37:0/1218531088 wait complete. 2024-09-17T04:33:46.963 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:33:46.963 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:33:46.983 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:33:47.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.101+0000 7ff9a95e4640 1 -- 172.21.15.37:0/2717525993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff990004800 msgr2=0x7ff990004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:47.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.101+0000 7ff9a95e4640 1 --2- 172.21.15.37:0/2717525993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff990004800 0x7ff990004c10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00bb20 tx=0x7ff990005830 comp rx=0 tx=0).stop 2024-09-17T04:33:47.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.101+0000 7ff9a37fe640 1 -- 172.21.15.37:0/2717525993 reap_dead start 2024-09-17T04:33:47.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.101+0000 7ff9a95e4640 1 -- 172.21.15.37:0/2717525993 shutdown_connections 2024-09-17T04:33:47.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.101+0000 7ff9a95e4640 1 -- 172.21.15.37:0/2717525993 >> 172.21.15.37:0/2717525993 conn(0x7ff9a406ddb0 msgr2=0x7ff9a406e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:47.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.102+0000 7ff9a95e4640 1 -- 172.21.15.37:0/2717525993 shutdown_connections 2024-09-17T04:33:47.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.102+0000 7ff9a95e4640 1 -- 172.21.15.37:0/2717525993 wait complete. 2024-09-17T04:33:47.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.103+0000 7ff9a95e4640 1 Processor -- start 2024-09-17T04:33:47.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.103+0000 7ff9a95e4640 1 -- start start 2024-09-17T04:33:47.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.105+0000 7ff9a95e4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff990004800 0x7ff9a41f5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:47.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.105+0000 7ff9a95e4640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff9a41538f0 0x7ff9a414a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:47.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.105+0000 7ff9a95e4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a40b8d50 0x7ff9a40b9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:47.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.105+0000 7ff9a95e4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a40d3420 con 0x7ff990004800 2024-09-17T04:33:47.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.105+0000 7ff9a95e4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a40d0f10 con 0x7ff9a40b8d50 2024-09-17T04:33:47.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.105+0000 7ff9a95e4640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7ff9a40d35a0 con 0x7ff9a41538f0 2024-09-17T04:33:47.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.105+0000 7ff9a2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff990004800 0x7ff9a41f5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:47.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.105+0000 7ff9a2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff990004800 0x7ff9a41f5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49264/0 (socket says 172.21.15.37:49264) 2024-09-17T04:33:47.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.105+0000 7ff9a2ffd640 1 -- 172.21.15.37:0/4069188066 learned_addr learned my addr 172.21.15.37:0/4069188066 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:47.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.105+0000 7ff9a27fc640 1 --2- 172.21.15.37:0/4069188066 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff9a41538f0 0x7ff9a414a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:47.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.105+0000 7ff9a37fe640 1 --2- 172.21.15.37:0/4069188066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a40b8d50 0x7ff9a40b9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:47.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.105+0000 7ff9a2ffd640 1 -- 172.21.15.37:0/4069188066 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff9a41538f0 msgr2=0x7ff9a414a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:47.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.105+0000 7ff9a2ffd640 1 --2- 172.21.15.37:0/4069188066 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff9a41538f0 0x7ff9a414a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.105+0000 7ff9a2ffd640 1 -- 172.21.15.37:0/4069188066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a40b8d50 msgr2=0x7ff9a40b9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:47.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.105+0000 7ff9a2ffd640 1 --2- 172.21.15.37:0/4069188066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a40b8d50 0x7ff9a40b9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.105+0000 7ff9a2ffd640 1 -- 172.21.15.37:0/4069188066 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a40b96a0 con 0x7ff990004800 2024-09-17T04:33:47.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.105+0000 7ff9a2ffd640 1 --2- 172.21.15.37:0/4069188066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff990004800 0x7ff9a41f5ae0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff994006310 tx=0x7ff9940067d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:47.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.106+0000 7ff99b7fe640 1 -- 172.21.15.37:0/4069188066 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff994015070 con 0x7ff990004800 2024-09-17T04:33:47.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.106+0000 7ff99b7fe640 1 -- 172.21.15.37:0/4069188066 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff99400fd60 con 0x7ff990004800 2024-09-17T04:33:47.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.106+0000 7ff9a95e4640 1 -- 172.21.15.37:0/4069188066 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a414ad70 con 0x7ff990004800 2024-09-17T04:33:47.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.106+0000 7ff99b7fe640 1 -- 172.21.15.37:0/4069188066 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff994011690 con 0x7ff990004800 2024-09-17T04:33:47.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.106+0000 7ff9a95e4640 1 -- 172.21.15.37:0/4069188066 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a415e790 con 0x7ff990004800 2024-09-17T04:33:47.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.107+0000 7ff9a95e4640 1 -- 172.21.15.37:0/4069188066 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9a40d0a00 con 0x7ff990004800 2024-09-17T04:33:47.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.107+0000 7ff99b7fe640 1 -- 172.21.15.37:0/4069188066 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff994011830 con 0x7ff990004800 2024-09-17T04:33:47.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.108+0000 7ff99b7fe640 1 --2- 172.21.15.37:0/4069188066 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff984078b90 0x7ff98407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:47.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.108+0000 7ff99b7fe640 1 -- 172.21.15.37:0/4069188066 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff99409bda0 con 0x7ff990004800 2024-09-17T04:33:47.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.108+0000 7ff9a27fc640 1 --2- 172.21.15.37:0/4069188066 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff984078b90 0x7ff98407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:47.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.109+0000 7ff9a27fc640 1 --2- 172.21.15.37:0/4069188066 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff984078b90 0x7ff98407b0a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff98c036000 tx=0x7ff98c0046c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:47.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.112+0000 7ff99b7fe640 1 -- 172.21.15.37:0/4069188066 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff9a40d0a00 con 0x7ff990004800 2024-09-17T04:33:47.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.310+0000 7ff9a95e4640 1 -- 172.21.15.37:0/4069188066 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff9a40d2ee0 con 0x7ff990004800 2024-09-17T04:33:47.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.311+0000 7ff99b7fe640 1 -- 172.21.15.37:0/4069188066 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7ff9a40b96a0 con 0x7ff990004800 2024-09-17T04:33:47.311 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-09-17T04:33:47.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.313+0000 7ff9a95e4640 1 -- 172.21.15.37:0/4069188066 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff984078b90 msgr2=0x7ff98407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:47.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.313+0000 7ff9a95e4640 1 --2- 172.21.15.37:0/4069188066 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff984078b90 0x7ff98407b0a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff98c036000 tx=0x7ff98c0046c0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.313+0000 7ff9a95e4640 1 -- 172.21.15.37:0/4069188066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff990004800 msgr2=0x7ff9a41f5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:47.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.313+0000 7ff9a95e4640 1 --2- 172.21.15.37:0/4069188066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff990004800 0x7ff9a41f5ae0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff994006310 tx=0x7ff9940067d0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.313+0000 7ff9a95e4640 1 -- 172.21.15.37:0/4069188066 shutdown_connections 2024-09-17T04:33:47.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.313+0000 7ff9a95e4640 1 --2- 172.21.15.37:0/4069188066 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff984078b90 0x7ff98407b0a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.313+0000 7ff9a95e4640 1 --2- 172.21.15.37:0/4069188066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a40b8d50 0x7ff9a40b9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.313+0000 7ff9a95e4640 1 --2- 172.21.15.37:0/4069188066 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff9a41538f0 0x7ff9a414a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.313+0000 7ff9a95e4640 1 --2- 172.21.15.37:0/4069188066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff990004800 0x7ff9a41f5ae0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.313+0000 7ff9a95e4640 1 -- 172.21.15.37:0/4069188066 >> 172.21.15.37:0/4069188066 conn(0x7ff9a406ddb0 msgr2=0x7ff9a40cef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:47.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.313+0000 7ff9a95e4640 1 -- 172.21.15.37:0/4069188066 shutdown_connections 2024-09-17T04:33:47.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.313+0000 7ff9a95e4640 1 -- 172.21.15.37:0/4069188066 wait complete. 2024-09-17T04:33:47.324 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi092.front.sepia.ceph.com... 2024-09-17T04:33:47.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:47.324 DEBUG:teuthology.orchestra.run.smithi092:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-09-17T04:33:47.352 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:33:47.353 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2024-09-17T04:33:47.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:33:47.353 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-17T04:33:47.409 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T04:33:47.409 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:33:47.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.526+0000 7fb320cfa640 1 -- 172.21.15.37:0/1988283882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb31c0cf8f0 msgr2=0x7fb31c15b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:47.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.526+0000 7fb320cfa640 1 --2- 172.21.15.37:0/1988283882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb31c0cf8f0 0x7fb31c15b730 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb310005fd0 tx=0x7fb31002a540 comp rx=0 tx=0).stop 2024-09-17T04:33:47.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.526+0000 7fb320cfa640 1 -- 172.21.15.37:0/1988283882 shutdown_connections 2024-09-17T04:33:47.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.526+0000 7fb320cfa640 1 --2- 172.21.15.37:0/1988283882 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb31c15bc70 0x7fb31c15c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.526+0000 7fb320cfa640 1 --2- 172.21.15.37:0/1988283882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb31c0cf8f0 0x7fb31c15b730 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.526+0000 7fb320cfa640 1 -- 172.21.15.37:0/1988283882 >> 172.21.15.37:0/1988283882 conn(0x7fb31c069120 msgr2=0x7fb31c069530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:47.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.526+0000 7fb320cfa640 1 -- 172.21.15.37:0/1988283882 shutdown_connections 2024-09-17T04:33:47.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.526+0000 7fb320cfa640 1 -- 172.21.15.37:0/1988283882 wait complete. 2024-09-17T04:33:47.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.528+0000 7fb320cfa640 1 Processor -- start 2024-09-17T04:33:47.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.528+0000 7fb320cfa640 1 -- start start 2024-09-17T04:33:47.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.530+0000 7fb320cfa640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb31c15bc70 0x7fb31c1555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:47.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.530+0000 7fb320cfa640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb31c15d960 0x7fb31c154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:47.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.530+0000 7fb320cfa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb31c147db0 0x7fb31c1421f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:47.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.530+0000 7fb320cfa640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c0cd5e0 con 0x7fb31c147db0 2024-09-17T04:33:47.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.530+0000 7fb320cfa640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c0cd460 con 0x7fb31c15bc70 2024-09-17T04:33:47.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.530+0000 7fb320cfa640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fb31c15c5c0 con 0x7fb31c15d960 2024-09-17T04:33:47.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.530+0000 7fb31ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb31c147db0 0x7fb31c1421f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:47.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.530+0000 7fb31ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb31c147db0 0x7fb31c1421f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49278/0 (socket says 172.21.15.37:49278) 2024-09-17T04:33:47.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.530+0000 7fb31ad9d640 1 -- 172.21.15.37:0/1637651239 learned_addr learned my addr 172.21.15.37:0/1637651239 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:47.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.530+0000 7fb319d9b640 1 --2- 172.21.15.37:0/1637651239 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb31c15d960 0x7fb31c154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:47.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.530+0000 7fb31a59c640 1 --2- 172.21.15.37:0/1637651239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb31c15bc70 0x7fb31c1555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:47.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.530+0000 7fb31ad9d640 1 -- 172.21.15.37:0/1637651239 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb31c15d960 msgr2=0x7fb31c154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:47.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.530+0000 7fb31ad9d640 1 --2- 172.21.15.37:0/1637651239 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb31c15d960 0x7fb31c154410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.530+0000 7fb31ad9d640 1 -- 172.21.15.37:0/1637651239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb31c15bc70 msgr2=0x7fb31c1555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:47.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.530+0000 7fb31ad9d640 1 --2- 172.21.15.37:0/1637651239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb31c15bc70 0x7fb31c1555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.530+0000 7fb31ad9d640 1 -- 172.21.15.37:0/1637651239 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb31c15a6e0 con 0x7fb31c147db0 2024-09-17T04:33:47.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.530+0000 7fb31ad9d640 1 --2- 172.21.15.37:0/1637651239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb31c147db0 0x7fb31c1421f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb31400e990 tx=0x7fb31400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:47.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.531+0000 7fb30affd640 1 -- 172.21.15.37:0/1637651239 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb314004d20 con 0x7fb31c147db0 2024-09-17T04:33:47.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.531+0000 7fb30affd640 1 -- 172.21.15.37:0/1637651239 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb31400ca40 con 0x7fb31c147db0 2024-09-17T04:33:47.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.531+0000 7fb320cfa640 1 -- 172.21.15.37:0/1637651239 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb31c154ae0 con 0x7fb31c147db0 2024-09-17T04:33:47.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.531+0000 7fb30affd640 1 -- 172.21.15.37:0/1637651239 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb314005020 con 0x7fb31c147db0 2024-09-17T04:33:47.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.531+0000 7fb320cfa640 1 -- 172.21.15.37:0/1637651239 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb31c1550a0 con 0x7fb31c147db0 2024-09-17T04:33:47.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.532+0000 7fb320cfa640 1 -- 172.21.15.37:0/1637651239 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb31c15b730 con 0x7fb31c147db0 2024-09-17T04:33:47.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.532+0000 7fb30affd640 1 -- 172.21.15.37:0/1637651239 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb3140051c0 con 0x7fb31c147db0 2024-09-17T04:33:47.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.533+0000 7fb30affd640 1 --2- 172.21.15.37:0/1637651239 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb2fc078bc0 0x7fb2fc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:47.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.533+0000 7fb30affd640 1 -- 172.21.15.37:0/1637651239 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb31409b740 con 0x7fb31c147db0 2024-09-17T04:33:47.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.533+0000 7fb31a59c640 1 --2- 172.21.15.37:0/1637651239 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb2fc078bc0 0x7fb2fc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:47.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.534+0000 7fb31a59c640 1 --2- 172.21.15.37:0/1637651239 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb2fc078bc0 0x7fb2fc07b0d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb30c0015e0 tx=0x7fb30c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:47.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.537+0000 7fb30affd640 1 -- 172.21.15.37:0/1637651239 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb314016030 con 0x7fb31c147db0 2024-09-17T04:33:47.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.759+0000 7fb320cfa640 1 -- 172.21.15.37:0/1637651239 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb31c0d0f00 con 0x7fb31c147db0 2024-09-17T04:33:47.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.759+0000 7fb30affd640 1 -- 172.21.15.37:0/1637651239 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5217 (secure 0 0 0) 0x7fb314063a60 con 0x7fb31c147db0 2024-09-17T04:33:47.759 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:47.759 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"btime":"2024-09-17T04:33:22:694087+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4387,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/4285801054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4285801054},{"type":"v1","addr":"172.21.15.92:6836","nonce":4285801054}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4401,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/3514084301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3514084301},{"type":"v1","addr":"172.21.15.37:6834","nonce":3514084301}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4404,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3229527106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":3229527106},{"type":"v1","addr":"172.21.15.37:6835","nonce":3229527106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:19.050573+0000","modified":"2024-09-17T04:33:22.076946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4419":{"gid":4419,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2128554253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2128554253},{"type":"v1","addr":"172.21.15.92:6837","nonce":2128554253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4419,"qdb_cluster":[4419]},"id":1}]} 2024-09-17T04:33:47.759 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-09-17T04:33:47.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.761+0000 7fb320cfa640 1 -- 172.21.15.37:0/1637651239 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb2fc078bc0 msgr2=0x7fb2fc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:47.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.761+0000 7fb320cfa640 1 --2- 172.21.15.37:0/1637651239 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb2fc078bc0 0x7fb2fc07b0d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb30c0015e0 tx=0x7fb30c009040 comp rx=0 tx=0).stop 2024-09-17T04:33:47.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.761+0000 7fb320cfa640 1 -- 172.21.15.37:0/1637651239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb31c147db0 msgr2=0x7fb31c1421f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:47.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.761+0000 7fb320cfa640 1 --2- 172.21.15.37:0/1637651239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb31c147db0 0x7fb31c1421f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb31400e990 tx=0x7fb31400ee50 comp rx=0 tx=0).stop 2024-09-17T04:33:47.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.761+0000 7fb320cfa640 1 -- 172.21.15.37:0/1637651239 shutdown_connections 2024-09-17T04:33:47.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.761+0000 7fb320cfa640 1 --2- 172.21.15.37:0/1637651239 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb2fc078bc0 0x7fb2fc07b0d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.761+0000 7fb320cfa640 1 --2- 172.21.15.37:0/1637651239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb31c147db0 0x7fb31c1421f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.761+0000 7fb320cfa640 1 --2- 172.21.15.37:0/1637651239 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb31c15d960 0x7fb31c154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.761+0000 7fb320cfa640 1 --2- 172.21.15.37:0/1637651239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb31c15bc70 0x7fb31c1555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.761+0000 7fb320cfa640 1 -- 172.21.15.37:0/1637651239 >> 172.21.15.37:0/1637651239 conn(0x7fb31c069120 msgr2=0x7fb31c0b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:47.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.761+0000 7fb320cfa640 1 -- 172.21.15.37:0/1637651239 shutdown_connections 2024-09-17T04:33:47.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.761+0000 7fb320cfa640 1 -- 172.21.15.37:0/1637651239 wait complete. 2024-09-17T04:33:47.773 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:33:47.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.889+0000 7f52a2c20640 1 -- 172.21.15.37:0/1527749373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529c1608e0 msgr2=0x7f529c0ccfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:47.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.889+0000 7f52a2c20640 1 --2- 172.21.15.37:0/1527749373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529c1608e0 0x7f529c0ccfb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f529c0cc890 tx=0x7f528c02a530 comp rx=0 tx=0).stop 2024-09-17T04:33:47.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.890+0000 7f52a2c20640 1 -- 172.21.15.37:0/1527749373 shutdown_connections 2024-09-17T04:33:47.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.890+0000 7f52a2c20640 1 --2- 172.21.15.37:0/1527749373 >> v2:172.21.15.92:3300/0 conn(0x7f529c0ce950 0x7f529c0ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.890+0000 7f52a2c20640 1 --2- 172.21.15.37:0/1527749373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529c1608e0 0x7f529c0ccfb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.890+0000 7f52a2c20640 1 -- 172.21.15.37:0/1527749373 >> 172.21.15.37:0/1527749373 conn(0x7f529c070ff0 msgr2=0x7f529c071400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:47.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.890+0000 7f52a2c20640 1 -- 172.21.15.37:0/1527749373 shutdown_connections 2024-09-17T04:33:47.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.890+0000 7f52a2c20640 1 -- 172.21.15.37:0/1527749373 wait complete. 2024-09-17T04:33:47.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.891+0000 7f52a2c20640 1 Processor -- start 2024-09-17T04:33:47.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.892+0000 7f52a2c20640 1 -- start start 2024-09-17T04:33:47.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.893+0000 7f52a2c20640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529c0ce950 0x7f529c0c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:47.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.893+0000 7f52a2c20640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f529c15d860 0x7f529c0d0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:47.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.893+0000 7f52a2c20640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f529c1608e0 0x7f529c0d13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:47.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.893+0000 7f52a2c20640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f529c153200 con 0x7f529c0ce950 2024-09-17T04:33:47.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.893+0000 7f52a2c20640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f529c153080 con 0x7f529c15d860 2024-09-17T04:33:47.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.893+0000 7f52a2c20640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f529c153380 con 0x7f529c1608e0 2024-09-17T04:33:47.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.893+0000 7f52a09bc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529c0ce950 0x7f529c0c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:47.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.893+0000 7f52a09bc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529c0ce950 0x7f529c0c8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49288/0 (socket says 172.21.15.37:49288) 2024-09-17T04:33:47.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.893+0000 7f52a09bc640 1 -- 172.21.15.37:0/1519805007 learned_addr learned my addr 172.21.15.37:0/1519805007 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:47.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.893+0000 7f5293fff640 1 --2- 172.21.15.37:0/1519805007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f529c15d860 0x7f529c0d0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:47.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.893+0000 7f52a11bd640 1 --2- 172.21.15.37:0/1519805007 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f529c1608e0 0x7f529c0d13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:47.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.893+0000 7f52a09bc640 1 -- 172.21.15.37:0/1519805007 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f529c1608e0 msgr2=0x7f529c0d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:47.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.893+0000 7f52a09bc640 1 --2- 172.21.15.37:0/1519805007 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f529c1608e0 0x7f529c0d13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.893+0000 7f52a09bc640 1 -- 172.21.15.37:0/1519805007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f529c15d860 msgr2=0x7f529c0d0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:47.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.893+0000 7f52a09bc640 1 --2- 172.21.15.37:0/1519805007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f529c15d860 0x7f529c0d0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:47.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.893+0000 7f52a09bc640 1 -- 172.21.15.37:0/1519805007 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529c1f33f0 con 0x7f529c0ce950 2024-09-17T04:33:47.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.893+0000 7f5293fff640 1 --2- 172.21.15.37:0/1519805007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f529c15d860 0x7f529c0d0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:47.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.893+0000 7f52a11bd640 1 --2- 172.21.15.37:0/1519805007 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f529c1608e0 0x7f529c0d13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:47.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.894+0000 7f52a09bc640 1 --2- 172.21.15.37:0/1519805007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529c0ce950 0x7f529c0c8700 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f529800c3c0 tx=0x7f529800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:47.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.894+0000 7f52917fa640 1 -- 172.21.15.37:0/1519805007 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298017070 con 0x7f529c0ce950 2024-09-17T04:33:47.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.894+0000 7f52917fa640 1 -- 172.21.15.37:0/1519805007 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52980035f0 con 0x7f529c0ce950 2024-09-17T04:33:47.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.894+0000 7f52a2c20640 1 -- 172.21.15.37:0/1519805007 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f529c1f36d0 con 0x7f529c0ce950 2024-09-17T04:33:47.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.894+0000 7f52917fa640 1 -- 172.21.15.37:0/1519805007 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298003a80 con 0x7f529c0ce950 2024-09-17T04:33:47.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.894+0000 7f52a2c20640 1 -- 172.21.15.37:0/1519805007 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f529c1f3c90 con 0x7f529c0ce950 2024-09-17T04:33:47.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.895+0000 7f52a2c20640 1 -- 172.21.15.37:0/1519805007 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f529c153860 con 0x7f529c0ce950 2024-09-17T04:33:47.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.896+0000 7f52917fa640 1 -- 172.21.15.37:0/1519805007 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f529801a070 con 0x7f529c0ce950 2024-09-17T04:33:47.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.896+0000 7f52917fa640 1 --2- 172.21.15.37:0/1519805007 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f5288078b90 0x7f528807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:47.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.896+0000 7f52917fa640 1 -- 172.21.15.37:0/1519805007 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f529809a0b0 con 0x7f529c0ce950 2024-09-17T04:33:47.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.897+0000 7f5293fff640 1 --2- 172.21.15.37:0/1519805007 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f5288078b90 0x7f528807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:47.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.897+0000 7f5293fff640 1 --2- 172.21.15.37:0/1519805007 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f5288078b90 0x7f528807b0a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f528c034000 tx=0x7f528c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:47.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:47.901+0000 7f52917fa640 1 -- 172.21.15.37:0/1519805007 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f5298062470 con 0x7f529c0ce950 2024-09-17T04:33:48.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.128+0000 7f52a2c20640 1 -- 172.21.15.37:0/1519805007 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f529c15ec30 con 0x7f529c0ce950 2024-09-17T04:33:48.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.129+0000 7f52917fa640 1 -- 172.21.15.37:0/1519805007 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5217 (secure 0 0 0) 0x7f5298061d70 con 0x7f529c0ce950 2024-09-17T04:33:48.129 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:48.129 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"btime":"2024-09-17T04:33:22:694087+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4387,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/4285801054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4285801054},{"type":"v1","addr":"172.21.15.92:6836","nonce":4285801054}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4401,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/3514084301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3514084301},{"type":"v1","addr":"172.21.15.37:6834","nonce":3514084301}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4404,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3229527106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":3229527106},{"type":"v1","addr":"172.21.15.37:6835","nonce":3229527106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:19.050573+0000","modified":"2024-09-17T04:33:22.076946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4419":{"gid":4419,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2128554253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2128554253},{"type":"v1","addr":"172.21.15.92:6837","nonce":2128554253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4419,"qdb_cluster":[4419]},"id":1}]} 2024-09-17T04:33:48.129 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-09-17T04:33:48.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.131+0000 7f52a2c20640 1 -- 172.21.15.37:0/1519805007 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f5288078b90 msgr2=0x7f528807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:48.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.131+0000 7f52a2c20640 1 --2- 172.21.15.37:0/1519805007 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f5288078b90 0x7f528807b0a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f528c034000 tx=0x7f528c004730 comp rx=0 tx=0).stop 2024-09-17T04:33:48.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.131+0000 7f52a2c20640 1 -- 172.21.15.37:0/1519805007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529c0ce950 msgr2=0x7f529c0c8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:48.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.131+0000 7f52a2c20640 1 --2- 172.21.15.37:0/1519805007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529c0ce950 0x7f529c0c8700 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f529800c3c0 tx=0x7f529800c880 comp rx=0 tx=0).stop 2024-09-17T04:33:48.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.131+0000 7f52a2c20640 1 -- 172.21.15.37:0/1519805007 shutdown_connections 2024-09-17T04:33:48.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.131+0000 7f52a2c20640 1 --2- 172.21.15.37:0/1519805007 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f5288078b90 0x7f528807b0a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.131+0000 7f52a2c20640 1 --2- 172.21.15.37:0/1519805007 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f529c1608e0 0x7f529c0d13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.131+0000 7f52a2c20640 1 --2- 172.21.15.37:0/1519805007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f529c15d860 0x7f529c0d0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.131+0000 7f52a2c20640 1 --2- 172.21.15.37:0/1519805007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529c0ce950 0x7f529c0c8700 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.131+0000 7f52a2c20640 1 -- 172.21.15.37:0/1519805007 >> 172.21.15.37:0/1519805007 conn(0x7f529c070ff0 msgr2=0x7f529c15c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:48.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.131+0000 7f52a2c20640 1 -- 172.21.15.37:0/1519805007 shutdown_connections 2024-09-17T04:33:48.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.131+0000 7f52a2c20640 1 -- 172.21.15.37:0/1519805007 wait complete. 2024-09-17T04:33:48.142 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:33:48.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.260+0000 7fc6099eb640 1 -- 172.21.15.37:0/1917380003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc60415d960 msgr2=0x7fc6040cf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:48.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.260+0000 7fc6099eb640 1 --2- 172.21.15.37:0/1917380003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc60415d960 0x7fc6040cf1e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc6040b4340 tx=0x7fc5f402a4a0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.260+0000 7fc6099eb640 1 -- 172.21.15.37:0/1917380003 shutdown_connections 2024-09-17T04:33:48.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.260+0000 7fc6099eb640 1 --2- 172.21.15.37:0/1917380003 >> v2:172.21.15.92:3300/0 conn(0x7fc60415c740 0x7fc604160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.260+0000 7fc6099eb640 1 --2- 172.21.15.37:0/1917380003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc60415d960 0x7fc6040cf1e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.260+0000 7fc6099eb640 1 -- 172.21.15.37:0/1917380003 >> 172.21.15.37:0/1917380003 conn(0x7fc604069120 msgr2=0x7fc604069530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:48.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.260+0000 7fc6099eb640 1 -- 172.21.15.37:0/1917380003 shutdown_connections 2024-09-17T04:33:48.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.260+0000 7fc6099eb640 1 -- 172.21.15.37:0/1917380003 wait complete. 2024-09-17T04:33:48.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.262+0000 7fc6099eb640 1 Processor -- start 2024-09-17T04:33:48.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.262+0000 7fc6099eb640 1 -- start start 2024-09-17T04:33:48.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.264+0000 7fc6099eb640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc6040cf720 0x7fc604159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:48.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.264+0000 7fc6099eb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc60415c740 0x7fc60406eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:48.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.264+0000 7fc6099eb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc60415d960 0x7fc60406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:48.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.264+0000 7fc6099eb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc6041573f0 con 0x7fc60415d960 2024-09-17T04:33:48.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.264+0000 7fc6099eb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc604157270 con 0x7fc60415c740 2024-09-17T04:33:48.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.264+0000 7fc6099eb640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fc6040cd460 con 0x7fc6040cf720 2024-09-17T04:33:48.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.264+0000 7fc6037fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc60415d960 0x7fc60406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:48.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.264+0000 7fc6037fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc60415d960 0x7fc60406f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49308/0 (socket says 172.21.15.37:49308) 2024-09-17T04:33:48.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.264+0000 7fc6037fe640 1 -- 172.21.15.37:0/1391336448 learned_addr learned my addr 172.21.15.37:0/1391336448 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:48.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.264+0000 7fc602ffd640 1 --2- 172.21.15.37:0/1391336448 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc6040cf720 0x7fc604159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:48.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.264+0000 7fc5fa7fc640 1 --2- 172.21.15.37:0/1391336448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc60415c740 0x7fc60406eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:48.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.264+0000 7fc6037fe640 1 -- 172.21.15.37:0/1391336448 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc6040cf720 msgr2=0x7fc604159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:48.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.264+0000 7fc6037fe640 1 --2- 172.21.15.37:0/1391336448 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc6040cf720 0x7fc604159f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.264+0000 7fc6037fe640 1 -- 172.21.15.37:0/1391336448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc60415c740 msgr2=0x7fc60406eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:48.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.264+0000 7fc6037fe640 1 --2- 172.21.15.37:0/1391336448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc60415c740 0x7fc60406eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.264+0000 7fc6037fe640 1 -- 172.21.15.37:0/1391336448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6041e5c90 con 0x7fc60415d960 2024-09-17T04:33:48.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.264+0000 7fc6037fe640 1 --2- 172.21.15.37:0/1391336448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc60415d960 0x7fc60406f0e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00cab0 tx=0x7fc5fc00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:48.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.264+0000 7fc600ff9640 1 -- 172.21.15.37:0/1391336448 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc007d90 con 0x7fc60415d960 2024-09-17T04:33:48.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.265+0000 7fc600ff9640 1 -- 172.21.15.37:0/1391336448 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5fc00f540 con 0x7fc60415d960 2024-09-17T04:33:48.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.265+0000 7fc6099eb640 1 -- 172.21.15.37:0/1391336448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc60406e0c0 con 0x7fc60415d960 2024-09-17T04:33:48.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.265+0000 7fc600ff9640 1 -- 172.21.15.37:0/1391336448 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc00f990 con 0x7fc60415d960 2024-09-17T04:33:48.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.265+0000 7fc6099eb640 1 -- 172.21.15.37:0/1391336448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc60406e680 con 0x7fc60415d960 2024-09-17T04:33:48.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.266+0000 7fc6099eb640 1 -- 172.21.15.37:0/1391336448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6040cf1e0 con 0x7fc60415d960 2024-09-17T04:33:48.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.266+0000 7fc600ff9640 1 -- 172.21.15.37:0/1391336448 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc5fc00fb30 con 0x7fc60415d960 2024-09-17T04:33:48.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.267+0000 7fc600ff9640 1 --2- 172.21.15.37:0/1391336448 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc5e4078bc0 0x7fc5e407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:48.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.267+0000 7fc600ff9640 1 -- 172.21.15.37:0/1391336448 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc5fc09b640 con 0x7fc60415d960 2024-09-17T04:33:48.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.267+0000 7fc602ffd640 1 --2- 172.21.15.37:0/1391336448 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc5e4078bc0 0x7fc5e407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:48.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.268+0000 7fc602ffd640 1 --2- 172.21.15.37:0/1391336448 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc5e4078bc0 0x7fc5e407b0d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4034000 tx=0x7fc5f4004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:48.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.271+0000 7fc600ff9640 1 -- 172.21.15.37:0/1391336448 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc5fc014070 con 0x7fc60415d960 2024-09-17T04:33:48.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.462+0000 7fc6099eb640 1 -- 172.21.15.37:0/1391336448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc6040d2300 con 0x7fc60415d960 2024-09-17T04:33:48.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.463+0000 7fc600ff9640 1 -- 172.21.15.37:0/1391336448 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16605 (secure 0 0 0) 0x7fc5fc00fe20 con 0x7fc60415d960 2024-09-17T04:33:48.463 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:48.463 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":22,"fsid":"a908a603-430c-4209-8294-0c3460b35f42","created":"2024-09-17T04:32:54.139221+0000","modified":"2024-09-17T04:33:20.677493+0000","last_up_change":"2024-09-17T04:33:06.922210+0000","last_in_change":"2024-09-17T04:32:57.754118+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:33:08.461907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:33:16.958785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:18.013768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ea4b8d1-0d22-4dd9-b913-9df585ef24ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6809","nonce":2283587593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6811","nonce":2283587593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6815","nonce":2283587593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6813","nonce":2283587593}]},"public_addr":"172.21.15.37:6809/2283587593","cluster_addr":"172.21.15.37:6811/2283587593","heartbeat_back_addr":"172.21.15.37:6815/2283587593","heartbeat_front_addr":"172.21.15.37:6813/2283587593","state":["exists","up"]},{"osd":1,"uuid":"ac745a33-e8ad-4083-9c18-71bf81982b76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6825","nonce":2657955935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6827","nonce":2657955935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6831","nonce":2657955935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6829","nonce":2657955935}]},"public_addr":"172.21.15.37:6825/2657955935","cluster_addr":"172.21.15.37:6827/2657955935","heartbeat_back_addr":"172.21.15.37:6831/2657955935","heartbeat_front_addr":"172.21.15.37:6829/2657955935","state":["exists","up"]},{"osd":2,"uuid":"3c63f3d8-132a-4735-b0bc-d64313a46503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6817","nonce":579720176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6819","nonce":579720176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6823","nonce":579720176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6821","nonce":579720176}]},"public_addr":"172.21.15.37:6817/579720176","cluster_addr":"172.21.15.37:6819/579720176","heartbeat_back_addr":"172.21.15.37:6823/579720176","heartbeat_front_addr":"172.21.15.37:6821/579720176","state":["exists","up"]},{"osd":3,"uuid":"4f8c6ca2-ea1d-4610-9127-52c4dd8bc8e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6801","nonce":809975826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6803","nonce":809975826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6807","nonce":809975826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6805","nonce":809975826}]},"public_addr":"172.21.15.37:6801/809975826","cluster_addr":"172.21.15.37:6803/809975826","heartbeat_back_addr":"172.21.15.37:6807/809975826","heartbeat_front_addr":"172.21.15.37:6805/809975826","state":["exists","up"]},{"osd":4,"uuid":"63d0c4ab-f115-47b4-9611-12a10153473a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6819","nonce":2509785752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6821","nonce":2509785752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6825","nonce":2509785752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6823","nonce":2509785752}]},"public_addr":"172.21.15.92:6819/2509785752","cluster_addr":"172.21.15.92:6821/2509785752","heartbeat_back_addr":"172.21.15.92:6825/2509785752","heartbeat_front_addr":"172.21.15.92:6823/2509785752","state":["exists","up"]},{"osd":5,"uuid":"4c56c04d-0c07-4f76-9d75-6335121310ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6827","nonce":248263832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6829","nonce":248263832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6833","nonce":248263832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6831","nonce":248263832}]},"public_addr":"172.21.15.92:6827/248263832","cluster_addr":"172.21.15.92:6829/248263832","heartbeat_back_addr":"172.21.15.92:6833/248263832","heartbeat_front_addr":"172.21.15.92:6831/248263832","state":["exists","up"]},{"osd":6,"uuid":"b30eae7c-096a-4d24-ac51-96b8b13ecaee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6811","nonce":4078365985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6813","nonce":4078365985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6817","nonce":4078365985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6815","nonce":4078365985}]},"public_addr":"172.21.15.92:6811/4078365985","cluster_addr":"172.21.15.92:6813/4078365985","heartbeat_back_addr":"172.21.15.92:6817/4078365985","heartbeat_front_addr":"172.21.15.92:6815/4078365985","state":["exists","up"]},{"osd":7,"uuid":"62f12540-d374-4a4c-9ff7-63b7304fe77f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6803","nonce":3444787223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6805","nonce":3444787223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6809","nonce":3444787223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6807","nonce":3444787223}]},"public_addr":"172.21.15.92:6803/3444787223","cluster_addr":"172.21.15.92:6805/3444787223","heartbeat_back_addr":"172.21.15.92:6809/3444787223","heartbeat_front_addr":"172.21.15.92:6807/3444787223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.006420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.111135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.103325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.072656+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.200989+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.060455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.774773+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.865778+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:33:48.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.465+0000 7fc6099eb640 1 -- 172.21.15.37:0/1391336448 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc5e4078bc0 msgr2=0x7fc5e407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:48.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.465+0000 7fc6099eb640 1 --2- 172.21.15.37:0/1391336448 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc5e4078bc0 0x7fc5e407b0d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4034000 tx=0x7fc5f4004730 comp rx=0 tx=0).stop 2024-09-17T04:33:48.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.465+0000 7fc6099eb640 1 -- 172.21.15.37:0/1391336448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc60415d960 msgr2=0x7fc60406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:48.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.465+0000 7fc6099eb640 1 --2- 172.21.15.37:0/1391336448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc60415d960 0x7fc60406f0e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00cab0 tx=0x7fc5fc00cf70 comp rx=0 tx=0).stop 2024-09-17T04:33:48.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.465+0000 7fc6099eb640 1 -- 172.21.15.37:0/1391336448 shutdown_connections 2024-09-17T04:33:48.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.465+0000 7fc6099eb640 1 --2- 172.21.15.37:0/1391336448 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc5e4078bc0 0x7fc5e407b0d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.465+0000 7fc6099eb640 1 --2- 172.21.15.37:0/1391336448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc60415d960 0x7fc60406f0e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.465+0000 7fc6099eb640 1 --2- 172.21.15.37:0/1391336448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc60415c740 0x7fc60406eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.465+0000 7fc6099eb640 1 --2- 172.21.15.37:0/1391336448 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc6040cf720 0x7fc604159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.465+0000 7fc6099eb640 1 -- 172.21.15.37:0/1391336448 >> 172.21.15.37:0/1391336448 conn(0x7fc604069120 msgr2=0x7fc604154fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:48.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.465+0000 7fc6099eb640 1 -- 172.21.15.37:0/1391336448 shutdown_connections 2024-09-17T04:33:48.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.465+0000 7fc6099eb640 1 -- 172.21.15.37:0/1391336448 wait complete. 2024-09-17T04:33:48.477 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T04:33:48.477 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:33:48.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.593+0000 7f9166bed640 1 -- 172.21.15.37:0/3003071104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f916015cb80 msgr2=0x7f91600b7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:48.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.593+0000 7f9166bed640 1 --2- 172.21.15.37:0/3003071104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f916015cb80 0x7f91600b7d50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9160073ac0 tx=0x7f915002a4d0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.593+0000 7f9166bed640 1 -- 172.21.15.37:0/3003071104 shutdown_connections 2024-09-17T04:33:48.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.593+0000 7f9166bed640 1 --2- 172.21.15.37:0/3003071104 >> v2:172.21.15.92:3300/0 conn(0x7f91600cd290 0x7f91600cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.593+0000 7f9166bed640 1 --2- 172.21.15.37:0/3003071104 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f91600cce80 0x7f91600ced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.593+0000 7f9166bed640 1 --2- 172.21.15.37:0/3003071104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f916015cb80 0x7f91600b7d50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.593+0000 7f9166bed640 1 -- 172.21.15.37:0/3003071104 >> 172.21.15.37:0/3003071104 conn(0x7f9160072a80 msgr2=0x7f9160072e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:48.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.593+0000 7f9166bed640 1 -- 172.21.15.37:0/3003071104 shutdown_connections 2024-09-17T04:33:48.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.593+0000 7f9166bed640 1 -- 172.21.15.37:0/3003071104 wait complete. 2024-09-17T04:33:48.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.595+0000 7f9166bed640 1 Processor -- start 2024-09-17T04:33:48.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.595+0000 7f9166bed640 1 -- start start 2024-09-17T04:33:48.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.596+0000 7f9166bed640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f91600cce80 0x7f916015c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:48.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.596+0000 7f9166bed640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91600cd290 0x7f91600d2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:48.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.596+0000 7f9166bed640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f916015cb80 0x7f91601f2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:48.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.596+0000 7f9166bed640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f91601534f0 con 0x7f91600cd290 2024-09-17T04:33:48.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.596+0000 7f9166bed640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9160153370 con 0x7f916015cb80 2024-09-17T04:33:48.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.596+0000 7f9166bed640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f9160153670 con 0x7f91600cce80 2024-09-17T04:33:48.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.597+0000 7f915ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91600cd290 0x7f91600d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:48.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.597+0000 7f915ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91600cd290 0x7f91600d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49330/0 (socket says 172.21.15.37:49330) 2024-09-17T04:33:48.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.597+0000 7f915ffff640 1 -- 172.21.15.37:0/3963606641 learned_addr learned my addr 172.21.15.37:0/3963606641 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:48.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.597+0000 7f9164989640 1 --2- 172.21.15.37:0/3963606641 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f91600cce80 0x7f916015c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:48.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.597+0000 7f916518a640 1 --2- 172.21.15.37:0/3963606641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f916015cb80 0x7f91601f2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:48.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.597+0000 7f915ffff640 1 -- 172.21.15.37:0/3963606641 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f91600cce80 msgr2=0x7f916015c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:48.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.597+0000 7f915ffff640 1 --2- 172.21.15.37:0/3963606641 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f91600cce80 0x7f916015c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.597+0000 7f915ffff640 1 -- 172.21.15.37:0/3963606641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f916015cb80 msgr2=0x7f91601f2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:48.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.597+0000 7f915ffff640 1 --2- 172.21.15.37:0/3963606641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f916015cb80 0x7f91601f2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.597+0000 7f915ffff640 1 -- 172.21.15.37:0/3963606641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9160157c80 con 0x7f91600cd290 2024-09-17T04:33:48.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.597+0000 7f915ffff640 1 --2- 172.21.15.37:0/3963606641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91600cd290 0x7f91600d2400 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f914800d660 tx=0x7f914800db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:48.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.597+0000 7f915d7fa640 1 -- 172.21.15.37:0/3963606641 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9148017070 con 0x7f91600cd290 2024-09-17T04:33:48.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.597+0000 7f915d7fa640 1 -- 172.21.15.37:0/3963606641 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9148005310 con 0x7f91600cd290 2024-09-17T04:33:48.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.597+0000 7f9166bed640 1 -- 172.21.15.37:0/3963606641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9160157f60 con 0x7f91600cd290 2024-09-17T04:33:48.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.597+0000 7f915d7fa640 1 -- 172.21.15.37:0/3963606641 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9148009820 con 0x7f91600cd290 2024-09-17T04:33:48.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.597+0000 7f9166bed640 1 -- 172.21.15.37:0/3963606641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91600d1c00 con 0x7f91600cd290 2024-09-17T04:33:48.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.598+0000 7f9166bed640 1 -- 172.21.15.37:0/3963606641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f916015fee0 con 0x7f91600cd290 2024-09-17T04:33:48.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.599+0000 7f915d7fa640 1 -- 172.21.15.37:0/3963606641 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f91480099c0 con 0x7f91600cd290 2024-09-17T04:33:48.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.600+0000 7f915d7fa640 1 --2- 172.21.15.37:0/3963606641 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9140078b90 0x7f914007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:48.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.600+0000 7f915d7fa640 1 -- 172.21.15.37:0/3963606641 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f914809acf0 con 0x7f91600cd290 2024-09-17T04:33:48.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.600+0000 7f9164989640 1 --2- 172.21.15.37:0/3963606641 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9140078b90 0x7f914007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:48.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.601+0000 7f9164989640 1 --2- 172.21.15.37:0/3963606641 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9140078b90 0x7f914007b0a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9150033000 tx=0x7f9150004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:48.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.605+0000 7f915d7fa640 1 -- 172.21.15.37:0/3963606641 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f916015fee0 con 0x7f91600cd290 2024-09-17T04:33:48.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.827+0000 7f9166bed640 1 -- 172.21.15.37:0/3963606641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91600b7d50 con 0x7f91600cd290 2024-09-17T04:33:48.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.827+0000 7f915d7fa640 1 -- 172.21.15.37:0/3963606641 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5217 (secure 0 0 0) 0x7f9148062b60 con 0x7f91600cd290 2024-09-17T04:33:48.827 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:48.827 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"btime":"2024-09-17T04:33:22:694087+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4387,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/4285801054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4285801054},{"type":"v1","addr":"172.21.15.92:6836","nonce":4285801054}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4401,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/3514084301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3514084301},{"type":"v1","addr":"172.21.15.37:6834","nonce":3514084301}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4404,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3229527106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":3229527106},{"type":"v1","addr":"172.21.15.37:6835","nonce":3229527106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:19.050573+0000","modified":"2024-09-17T04:33:22.076946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4419":{"gid":4419,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2128554253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2128554253},{"type":"v1","addr":"172.21.15.92:6837","nonce":2128554253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4419,"qdb_cluster":[4419]},"id":1}]} 2024-09-17T04:33:48.828 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-09-17T04:33:48.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.829+0000 7f9166bed640 1 -- 172.21.15.37:0/3963606641 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9140078b90 msgr2=0x7f914007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:48.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.829+0000 7f9166bed640 1 --2- 172.21.15.37:0/3963606641 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9140078b90 0x7f914007b0a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9150033000 tx=0x7f9150004730 comp rx=0 tx=0).stop 2024-09-17T04:33:48.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.829+0000 7f9166bed640 1 -- 172.21.15.37:0/3963606641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91600cd290 msgr2=0x7f91600d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:48.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.829+0000 7f9166bed640 1 --2- 172.21.15.37:0/3963606641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91600cd290 0x7f91600d2400 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f914800d660 tx=0x7f914800db20 comp rx=0 tx=0).stop 2024-09-17T04:33:48.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.829+0000 7f9166bed640 1 -- 172.21.15.37:0/3963606641 shutdown_connections 2024-09-17T04:33:48.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.829+0000 7f9166bed640 1 --2- 172.21.15.37:0/3963606641 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9140078b90 0x7f914007b0a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.829+0000 7f9166bed640 1 --2- 172.21.15.37:0/3963606641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f916015cb80 0x7f91601f2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.829+0000 7f9166bed640 1 --2- 172.21.15.37:0/3963606641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91600cd290 0x7f91600d2400 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.829+0000 7f9166bed640 1 --2- 172.21.15.37:0/3963606641 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f91600cce80 0x7f916015c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.829+0000 7f9166bed640 1 -- 172.21.15.37:0/3963606641 >> 172.21.15.37:0/3963606641 conn(0x7f9160072a80 msgr2=0x7f91600b5770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:48.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.829+0000 7f9166bed640 1 -- 172.21.15.37:0/3963606641 shutdown_connections 2024-09-17T04:33:48.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.829+0000 7f9166bed640 1 -- 172.21.15.37:0/3963606641 wait complete. 2024-09-17T04:33:48.841 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:33:48.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.958+0000 7f937c438640 1 -- 172.21.15.37:0/396254553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93740d3fe0 msgr2=0x7f937415e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:48.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.958+0000 7f937c438640 1 --2- 172.21.15.37:0/396254553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93740d3fe0 0x7f937415e8e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9364005fd0 tx=0x7f936402a840 comp rx=0 tx=0).stop 2024-09-17T04:33:48.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.958+0000 7f937c438640 1 -- 172.21.15.37:0/396254553 shutdown_connections 2024-09-17T04:33:48.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.958+0000 7f937c438640 1 --2- 172.21.15.37:0/396254553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93740d3fe0 0x7f937415e8e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.958+0000 7f937c438640 1 -- 172.21.15.37:0/396254553 >> 172.21.15.37:0/396254553 conn(0x7f93740d44c0 msgr2=0x7f93740d48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:48.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.958+0000 7f937c438640 1 -- 172.21.15.37:0/396254553 shutdown_connections 2024-09-17T04:33:48.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.958+0000 7f937c438640 1 -- 172.21.15.37:0/396254553 wait complete. 2024-09-17T04:33:48.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.960+0000 7f937c438640 1 Processor -- start 2024-09-17T04:33:48.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.960+0000 7f937c438640 1 -- start start 2024-09-17T04:33:48.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.961+0000 7f937c438640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93740cef40 0x7f9374150460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:48.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.961+0000 7f937c438640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9374154570 0x7f9374161100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:48.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.961+0000 7f937c438640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9374161640 0x7f93741538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:48.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.961+0000 7f937c438640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f93740b8400 con 0x7f93740cef40 2024-09-17T04:33:48.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.961+0000 7f937c438640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f93740b8280 con 0x7f9374161640 2024-09-17T04:33:48.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.961+0000 7f937c438640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f93740b8580 con 0x7f9374154570 2024-09-17T04:33:48.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.962+0000 7f937a1d4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93740cef40 0x7f9374150460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:48.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.962+0000 7f937a1d4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93740cef40 0x7f9374150460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49342/0 (socket says 172.21.15.37:49342) 2024-09-17T04:33:48.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.962+0000 7f937a1d4640 1 -- 172.21.15.37:0/3409838758 learned_addr learned my addr 172.21.15.37:0/3409838758 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:48.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.962+0000 7f937a9d5640 1 --2- 172.21.15.37:0/3409838758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9374161640 0x7f93741538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:48.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.962+0000 7f93799d3640 1 --2- 172.21.15.37:0/3409838758 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9374154570 0x7f9374161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:48.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.962+0000 7f937a1d4640 1 -- 172.21.15.37:0/3409838758 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9374154570 msgr2=0x7f9374161100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:48.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.962+0000 7f937a1d4640 1 --2- 172.21.15.37:0/3409838758 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9374154570 0x7f9374161100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.962+0000 7f937a1d4640 1 -- 172.21.15.37:0/3409838758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9374161640 msgr2=0x7f93741538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:48.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.962+0000 7f937a1d4640 1 --2- 172.21.15.37:0/3409838758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9374161640 0x7f93741538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:48.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.962+0000 7f937a1d4640 1 -- 172.21.15.37:0/3409838758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9374150dd0 con 0x7f93740cef40 2024-09-17T04:33:48.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.962+0000 7f937a1d4640 1 --2- 172.21.15.37:0/3409838758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93740cef40 0x7f9374150460 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9364034040 tx=0x7f9364032cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:48.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.962+0000 7f9372ffd640 1 -- 172.21.15.37:0/3409838758 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9364040070 con 0x7f93740cef40 2024-09-17T04:33:48.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.962+0000 7f9372ffd640 1 -- 172.21.15.37:0/3409838758 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9364004560 con 0x7f93740cef40 2024-09-17T04:33:48.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.962+0000 7f937c438640 1 -- 172.21.15.37:0/3409838758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93741510b0 con 0x7f93740cef40 2024-09-17T04:33:48.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.962+0000 7f9372ffd640 1 -- 172.21.15.37:0/3409838758 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936403c420 con 0x7f93740cef40 2024-09-17T04:33:48.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.962+0000 7f937c438640 1 -- 172.21.15.37:0/3409838758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f937414fd80 con 0x7f93740cef40 2024-09-17T04:33:48.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.963+0000 7f937c438640 1 -- 172.21.15.37:0/3409838758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93740d3fe0 con 0x7f93740cef40 2024-09-17T04:33:48.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.964+0000 7f9372ffd640 1 -- 172.21.15.37:0/3409838758 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9364043070 con 0x7f93740cef40 2024-09-17T04:33:48.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.965+0000 7f9372ffd640 1 --2- 172.21.15.37:0/3409838758 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9360078b90 0x7f936007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:48.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.965+0000 7f9372ffd640 1 -- 172.21.15.37:0/3409838758 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f93640bbf30 con 0x7f93740cef40 2024-09-17T04:33:48.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.965+0000 7f93799d3640 1 --2- 172.21.15.37:0/3409838758 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9360078b90 0x7f936007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:48.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.966+0000 7f93799d3640 1 --2- 172.21.15.37:0/3409838758 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9360078b90 0x7f936007b0a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9368001eb0 tx=0x7f9368001950 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:48.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:48.969+0000 7f9372ffd640 1 -- 172.21.15.37:0/3409838758 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9364083db0 con 0x7f93740cef40 2024-09-17T04:33:49.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.193+0000 7f937c438640 1 -- 172.21.15.37:0/3409838758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f937406cd60 con 0x7f93740cef40 2024-09-17T04:33:49.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.194+0000 7f9372ffd640 1 -- 172.21.15.37:0/3409838758 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5217 (secure 0 0 0) 0x7f93640887e0 con 0x7f93740cef40 2024-09-17T04:33:49.194 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:49.194 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"btime":"2024-09-17T04:33:22:694087+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4387,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/4285801054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4285801054},{"type":"v1","addr":"172.21.15.92:6836","nonce":4285801054}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4401,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/3514084301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3514084301},{"type":"v1","addr":"172.21.15.37:6834","nonce":3514084301}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4404,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3229527106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":3229527106},{"type":"v1","addr":"172.21.15.37:6835","nonce":3229527106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:19.050573+0000","modified":"2024-09-17T04:33:22.076946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4419":{"gid":4419,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2128554253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2128554253},{"type":"v1","addr":"172.21.15.92:6837","nonce":2128554253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4419,"qdb_cluster":[4419]},"id":1}]} 2024-09-17T04:33:49.194 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-09-17T04:33:49.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.196+0000 7f937c438640 1 -- 172.21.15.37:0/3409838758 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9360078b90 msgr2=0x7f936007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:49.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.196+0000 7f937c438640 1 --2- 172.21.15.37:0/3409838758 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9360078b90 0x7f936007b0a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9368001eb0 tx=0x7f9368001950 comp rx=0 tx=0).stop 2024-09-17T04:33:49.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.196+0000 7f937c438640 1 -- 172.21.15.37:0/3409838758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93740cef40 msgr2=0x7f9374150460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:49.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.196+0000 7f937c438640 1 --2- 172.21.15.37:0/3409838758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93740cef40 0x7f9374150460 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9364034040 tx=0x7f9364032cf0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.196+0000 7f937c438640 1 -- 172.21.15.37:0/3409838758 shutdown_connections 2024-09-17T04:33:49.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.196+0000 7f937c438640 1 --2- 172.21.15.37:0/3409838758 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9360078b90 0x7f936007b0a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.196+0000 7f937c438640 1 --2- 172.21.15.37:0/3409838758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9374161640 0x7f93741538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.196+0000 7f937c438640 1 --2- 172.21.15.37:0/3409838758 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9374154570 0x7f9374161100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.196+0000 7f937c438640 1 --2- 172.21.15.37:0/3409838758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f93740cef40 0x7f9374150460 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.196+0000 7f937c438640 1 -- 172.21.15.37:0/3409838758 >> 172.21.15.37:0/3409838758 conn(0x7f93740d44c0 msgr2=0x7f93740b4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:49.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.196+0000 7f937c438640 1 -- 172.21.15.37:0/3409838758 shutdown_connections 2024-09-17T04:33:49.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.196+0000 7f937c438640 1 -- 172.21.15.37:0/3409838758 wait complete. 2024-09-17T04:33:49.207 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:33:49.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.325+0000 7f9cf3d62640 1 -- 172.21.15.37:0/3478800835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cec0acf80 msgr2=0x7f9cec0caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:49.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.325+0000 7f9cf3d62640 1 --2- 172.21.15.37:0/3478800835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cec0acf80 0x7f9cec0caa30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9cec070a70 tx=0x7f9ce802a560 comp rx=0 tx=0).stop 2024-09-17T04:33:49.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.325+0000 7f9cf3d62640 1 -- 172.21.15.37:0/3478800835 shutdown_connections 2024-09-17T04:33:49.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.325+0000 7f9cf3d62640 1 --2- 172.21.15.37:0/3478800835 >> v2:172.21.15.92:3300/0 conn(0x7f9cec0cf730 0x7f9cec06f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.325+0000 7f9cf3d62640 1 --2- 172.21.15.37:0/3478800835 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9cec0caf70 0x7f9cec0cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.325+0000 7f9cf3d62640 1 --2- 172.21.15.37:0/3478800835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cec0acf80 0x7f9cec0caa30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.325+0000 7f9cf3d62640 1 -- 172.21.15.37:0/3478800835 >> 172.21.15.37:0/3478800835 conn(0x7f9cec0d00b0 msgr2=0x7f9cec0d04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:49.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.325+0000 7f9cf3d62640 1 -- 172.21.15.37:0/3478800835 shutdown_connections 2024-09-17T04:33:49.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.325+0000 7f9cf3d62640 1 -- 172.21.15.37:0/3478800835 wait complete. 2024-09-17T04:33:49.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.327+0000 7f9cf3d62640 1 Processor -- start 2024-09-17T04:33:49.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.327+0000 7f9cf3d62640 1 -- start start 2024-09-17T04:33:49.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.328+0000 7f9cf3d62640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cec0acf80 0x7f9cec09aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:49.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.328+0000 7f9cf3d62640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9cec0caf70 0x7f9cec150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:49.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.328+0000 7f9cf3d62640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cec0cf730 0x7f9cec159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:49.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.328+0000 7f9cf3d62640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9cec06f550 con 0x7f9cec0acf80 2024-09-17T04:33:49.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.328+0000 7f9cf3d62640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9cec0cfd50 con 0x7f9cec0cf730 2024-09-17T04:33:49.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.328+0000 7f9cf3d62640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f9cec06f6d0 con 0x7f9cec0caf70 2024-09-17T04:33:49.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.328+0000 7f9cf1afe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cec0acf80 0x7f9cec09aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:49.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.328+0000 7f9cf1afe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cec0acf80 0x7f9cec09aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49352/0 (socket says 172.21.15.37:49352) 2024-09-17T04:33:49.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.328+0000 7f9cf1afe640 1 -- 172.21.15.37:0/3158663015 learned_addr learned my addr 172.21.15.37:0/3158663015 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:49.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.328+0000 7f9cf22ff640 1 --2- 172.21.15.37:0/3158663015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cec0cf730 0x7f9cec159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:49.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.328+0000 7f9cf12fd640 1 --2- 172.21.15.37:0/3158663015 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9cec0caf70 0x7f9cec150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:49.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.328+0000 7f9cf1afe640 1 -- 172.21.15.37:0/3158663015 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9cec0caf70 msgr2=0x7f9cec150430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:49.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.328+0000 7f9cf1afe640 1 --2- 172.21.15.37:0/3158663015 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9cec0caf70 0x7f9cec150430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.328+0000 7f9cf1afe640 1 -- 172.21.15.37:0/3158663015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cec0cf730 msgr2=0x7f9cec159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:49.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.328+0000 7f9cf1afe640 1 --2- 172.21.15.37:0/3158663015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cec0cf730 0x7f9cec159b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.328+0000 7f9cf1afe640 1 -- 172.21.15.37:0/3158663015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cec09af60 con 0x7f9cec0acf80 2024-09-17T04:33:49.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.329+0000 7f9cf1afe640 1 --2- 172.21.15.37:0/3158663015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cec0acf80 0x7f9cec09aa20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8034040 tx=0x7f9ce80039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:49.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.329+0000 7f9ce27fc640 1 -- 172.21.15.37:0/3158663015 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce803e030 con 0x7f9cec0acf80 2024-09-17T04:33:49.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.329+0000 7f9ce27fc640 1 -- 172.21.15.37:0/3158663015 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ce80047a0 con 0x7f9cec0acf80 2024-09-17T04:33:49.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.329+0000 7f9ce27fc640 1 -- 172.21.15.37:0/3158663015 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8035040 con 0x7f9cec0acf80 2024-09-17T04:33:49.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.329+0000 7f9cf3d62640 1 -- 172.21.15.37:0/3158663015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cec09d560 con 0x7f9cec0acf80 2024-09-17T04:33:49.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.329+0000 7f9cf3d62640 1 -- 172.21.15.37:0/3158663015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cec159060 con 0x7f9cec0acf80 2024-09-17T04:33:49.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.330+0000 7f9cf3d62640 1 -- 172.21.15.37:0/3158663015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cec06fbf0 con 0x7f9cec0acf80 2024-09-17T04:33:49.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.331+0000 7f9ce27fc640 1 -- 172.21.15.37:0/3158663015 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9ce8041030 con 0x7f9cec0acf80 2024-09-17T04:33:49.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.331+0000 7f9ce27fc640 1 --2- 172.21.15.37:0/3158663015 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9cd0078b90 0x7f9cd007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:49.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.331+0000 7f9ce27fc640 1 -- 172.21.15.37:0/3158663015 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9ce80bb3f0 con 0x7f9cec0acf80 2024-09-17T04:33:49.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.332+0000 7f9cf12fd640 1 --2- 172.21.15.37:0/3158663015 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9cd0078b90 0x7f9cd007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:49.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.332+0000 7f9cf12fd640 1 --2- 172.21.15.37:0/3158663015 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9cd0078b90 0x7f9cd007b0a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9cec09d4f0 tx=0x7f9cdc00b260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:49.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.336+0000 7f9ce27fc640 1 -- 172.21.15.37:0/3158663015 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9ce8083300 con 0x7f9cec0acf80 2024-09-17T04:33:49.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.524+0000 7f9cf3d62640 1 -- 172.21.15.37:0/3158663015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9cec0b13c0 con 0x7f9cec0acf80 2024-09-17T04:33:49.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.525+0000 7f9ce27fc640 1 -- 172.21.15.37:0/3158663015 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16605 (secure 0 0 0) 0x7f9ce8087d30 con 0x7f9cec0acf80 2024-09-17T04:33:49.525 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:49.525 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":22,"fsid":"a908a603-430c-4209-8294-0c3460b35f42","created":"2024-09-17T04:32:54.139221+0000","modified":"2024-09-17T04:33:20.677493+0000","last_up_change":"2024-09-17T04:33:06.922210+0000","last_in_change":"2024-09-17T04:32:57.754118+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:33:08.461907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:33:16.958785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:18.013768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ea4b8d1-0d22-4dd9-b913-9df585ef24ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6809","nonce":2283587593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6811","nonce":2283587593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6815","nonce":2283587593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6813","nonce":2283587593}]},"public_addr":"172.21.15.37:6809/2283587593","cluster_addr":"172.21.15.37:6811/2283587593","heartbeat_back_addr":"172.21.15.37:6815/2283587593","heartbeat_front_addr":"172.21.15.37:6813/2283587593","state":["exists","up"]},{"osd":1,"uuid":"ac745a33-e8ad-4083-9c18-71bf81982b76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6825","nonce":2657955935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6827","nonce":2657955935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6831","nonce":2657955935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6829","nonce":2657955935}]},"public_addr":"172.21.15.37:6825/2657955935","cluster_addr":"172.21.15.37:6827/2657955935","heartbeat_back_addr":"172.21.15.37:6831/2657955935","heartbeat_front_addr":"172.21.15.37:6829/2657955935","state":["exists","up"]},{"osd":2,"uuid":"3c63f3d8-132a-4735-b0bc-d64313a46503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6817","nonce":579720176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6819","nonce":579720176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6823","nonce":579720176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6821","nonce":579720176}]},"public_addr":"172.21.15.37:6817/579720176","cluster_addr":"172.21.15.37:6819/579720176","heartbeat_back_addr":"172.21.15.37:6823/579720176","heartbeat_front_addr":"172.21.15.37:6821/579720176","state":["exists","up"]},{"osd":3,"uuid":"4f8c6ca2-ea1d-4610-9127-52c4dd8bc8e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6801","nonce":809975826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6803","nonce":809975826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6807","nonce":809975826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6805","nonce":809975826}]},"public_addr":"172.21.15.37:6801/809975826","cluster_addr":"172.21.15.37:6803/809975826","heartbeat_back_addr":"172.21.15.37:6807/809975826","heartbeat_front_addr":"172.21.15.37:6805/809975826","state":["exists","up"]},{"osd":4,"uuid":"63d0c4ab-f115-47b4-9611-12a10153473a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6819","nonce":2509785752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6821","nonce":2509785752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6825","nonce":2509785752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6823","nonce":2509785752}]},"public_addr":"172.21.15.92:6819/2509785752","cluster_addr":"172.21.15.92:6821/2509785752","heartbeat_back_addr":"172.21.15.92:6825/2509785752","heartbeat_front_addr":"172.21.15.92:6823/2509785752","state":["exists","up"]},{"osd":5,"uuid":"4c56c04d-0c07-4f76-9d75-6335121310ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6827","nonce":248263832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6829","nonce":248263832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6833","nonce":248263832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6831","nonce":248263832}]},"public_addr":"172.21.15.92:6827/248263832","cluster_addr":"172.21.15.92:6829/248263832","heartbeat_back_addr":"172.21.15.92:6833/248263832","heartbeat_front_addr":"172.21.15.92:6831/248263832","state":["exists","up"]},{"osd":6,"uuid":"b30eae7c-096a-4d24-ac51-96b8b13ecaee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6811","nonce":4078365985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6813","nonce":4078365985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6817","nonce":4078365985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6815","nonce":4078365985}]},"public_addr":"172.21.15.92:6811/4078365985","cluster_addr":"172.21.15.92:6813/4078365985","heartbeat_back_addr":"172.21.15.92:6817/4078365985","heartbeat_front_addr":"172.21.15.92:6815/4078365985","state":["exists","up"]},{"osd":7,"uuid":"62f12540-d374-4a4c-9ff7-63b7304fe77f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6803","nonce":3444787223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6805","nonce":3444787223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6809","nonce":3444787223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6807","nonce":3444787223}]},"public_addr":"172.21.15.92:6803/3444787223","cluster_addr":"172.21.15.92:6805/3444787223","heartbeat_back_addr":"172.21.15.92:6809/3444787223","heartbeat_front_addr":"172.21.15.92:6807/3444787223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.006420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.111135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.103325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.072656+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.200989+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.060455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.774773+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.865778+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:33:49.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.527+0000 7f9cf3d62640 1 -- 172.21.15.37:0/3158663015 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9cd0078b90 msgr2=0x7f9cd007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:49.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.527+0000 7f9cf3d62640 1 --2- 172.21.15.37:0/3158663015 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9cd0078b90 0x7f9cd007b0a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9cec09d4f0 tx=0x7f9cdc00b260 comp rx=0 tx=0).stop 2024-09-17T04:33:49.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.527+0000 7f9cf3d62640 1 -- 172.21.15.37:0/3158663015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cec0acf80 msgr2=0x7f9cec09aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:49.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.527+0000 7f9cf3d62640 1 --2- 172.21.15.37:0/3158663015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cec0acf80 0x7f9cec09aa20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8034040 tx=0x7f9ce80039e0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.527+0000 7f9cf3d62640 1 -- 172.21.15.37:0/3158663015 shutdown_connections 2024-09-17T04:33:49.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.527+0000 7f9cf3d62640 1 --2- 172.21.15.37:0/3158663015 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9cd0078b90 0x7f9cd007b0a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.527+0000 7f9cf3d62640 1 --2- 172.21.15.37:0/3158663015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cec0cf730 0x7f9cec159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.527+0000 7f9cf3d62640 1 --2- 172.21.15.37:0/3158663015 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9cec0caf70 0x7f9cec150430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.527+0000 7f9cf3d62640 1 --2- 172.21.15.37:0/3158663015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cec0acf80 0x7f9cec09aa20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.527+0000 7f9cf3d62640 1 -- 172.21.15.37:0/3158663015 >> 172.21.15.37:0/3158663015 conn(0x7f9cec0d00b0 msgr2=0x7f9cec0b4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:49.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.527+0000 7f9cf3d62640 1 -- 172.21.15.37:0/3158663015 shutdown_connections 2024-09-17T04:33:49.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.527+0000 7f9cf3d62640 1 -- 172.21.15.37:0/3158663015 wait complete. 2024-09-17T04:33:49.539 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T04:33:49.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.657+0000 7f9422cb4640 1 -- 172.21.15.37:0/1458301731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f941c0cf8f0 msgr2=0x7f941c15b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:49.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.657+0000 7f9422cb4640 1 --2- 172.21.15.37:0/1458301731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f941c0cf8f0 0x7f941c15b730 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f941c0b4340 tx=0x7f940402a980 comp rx=0 tx=0).stop 2024-09-17T04:33:49.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.657+0000 7f9422cb4640 1 -- 172.21.15.37:0/1458301731 shutdown_connections 2024-09-17T04:33:49.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.657+0000 7f9422cb4640 1 --2- 172.21.15.37:0/1458301731 >> v2:172.21.15.92:3300/0 conn(0x7f941c15bc70 0x7f941c15c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.657+0000 7f9422cb4640 1 --2- 172.21.15.37:0/1458301731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f941c0cf8f0 0x7f941c15b730 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.657+0000 7f9422cb4640 1 -- 172.21.15.37:0/1458301731 >> 172.21.15.37:0/1458301731 conn(0x7f941c069120 msgr2=0x7f941c069530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:49.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.657+0000 7f9422cb4640 1 -- 172.21.15.37:0/1458301731 shutdown_connections 2024-09-17T04:33:49.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.657+0000 7f9422cb4640 1 -- 172.21.15.37:0/1458301731 wait complete. 2024-09-17T04:33:49.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.659+0000 7f9422cb4640 1 Processor -- start 2024-09-17T04:33:49.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.659+0000 7f9422cb4640 1 -- start start 2024-09-17T04:33:49.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.661+0000 7f9422cb4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f941c15bc70 0x7f941c156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:49.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.661+0000 7f9422cb4640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f941c15d960 0x7f941c1e7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:49.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.661+0000 7f9422cb4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f941c0b2f20 0x7f941c159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:49.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.661+0000 7f9422cb4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f941c0cd5e0 con 0x7f941c15bc70 2024-09-17T04:33:49.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.661+0000 7f9422cb4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f941c0cd460 con 0x7f941c0b2f20 2024-09-17T04:33:49.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.661+0000 7f9422cb4640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f941c15c5c0 con 0x7f941c15d960 2024-09-17T04:33:49.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.661+0000 7f9420a50640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f941c15bc70 0x7f941c156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:49.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.661+0000 7f9420a50640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f941c15bc70 0x7f941c156260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49374/0 (socket says 172.21.15.37:49374) 2024-09-17T04:33:49.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.661+0000 7f9420a50640 1 -- 172.21.15.37:0/3718056905 learned_addr learned my addr 172.21.15.37:0/3718056905 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:49.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.661+0000 7f941bfff640 1 --2- 172.21.15.37:0/3718056905 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f941c15d960 0x7f941c1e7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:49.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.661+0000 7f9421251640 1 --2- 172.21.15.37:0/3718056905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f941c0b2f20 0x7f941c159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:49.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.661+0000 7f9420a50640 1 -- 172.21.15.37:0/3718056905 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f941c15d960 msgr2=0x7f941c1e7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:49.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.661+0000 7f9420a50640 1 --2- 172.21.15.37:0/3718056905 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f941c15d960 0x7f941c1e7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.661+0000 7f9420a50640 1 -- 172.21.15.37:0/3718056905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f941c0b2f20 msgr2=0x7f941c159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:49.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.661+0000 7f9420a50640 1 --2- 172.21.15.37:0/3718056905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f941c0b2f20 0x7f941c159650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.661+0000 7f9420a50640 1 -- 172.21.15.37:0/3718056905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f941c1e7f90 con 0x7f941c15bc70 2024-09-17T04:33:49.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.661+0000 7f941bfff640 1 --2- 172.21.15.37:0/3718056905 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f941c15d960 0x7f941c1e7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:49.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.661+0000 7f9421251640 1 --2- 172.21.15.37:0/3718056905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f941c0b2f20 0x7f941c159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:49.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.661+0000 7f9420a50640 1 --2- 172.21.15.37:0/3718056905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f941c15bc70 0x7f941c156260 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f940c00b420 tx=0x7f940c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:49.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.661+0000 7f94197fa640 1 -- 172.21.15.37:0/3718056905 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c011830 con 0x7f941c15bc70 2024-09-17T04:33:49.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.661+0000 7f94197fa640 1 -- 172.21.15.37:0/3718056905 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f940c016070 con 0x7f941c15bc70 2024-09-17T04:33:49.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.661+0000 7f9422cb4640 1 -- 172.21.15.37:0/3718056905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f941c1e50f0 con 0x7f941c15bc70 2024-09-17T04:33:49.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.661+0000 7f94197fa640 1 -- 172.21.15.37:0/3718056905 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c011d60 con 0x7f941c15bc70 2024-09-17T04:33:49.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.661+0000 7f9422cb4640 1 -- 172.21.15.37:0/3718056905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f941c1e56b0 con 0x7f941c15bc70 2024-09-17T04:33:49.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.663+0000 7f9422cb4640 1 -- 172.21.15.37:0/3718056905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f941c15b730 con 0x7f941c15bc70 2024-09-17T04:33:49.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.663+0000 7f94197fa640 1 -- 172.21.15.37:0/3718056905 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f940c01b460 con 0x7f941c15bc70 2024-09-17T04:33:49.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.664+0000 7f94197fa640 1 --2- 172.21.15.37:0/3718056905 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f93fc078b90 0x7f93fc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:49.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.664+0000 7f94197fa640 1 -- 172.21.15.37:0/3718056905 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f940c09a830 con 0x7f941c15bc70 2024-09-17T04:33:49.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.664+0000 7f941bfff640 1 --2- 172.21.15.37:0/3718056905 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f93fc078b90 0x7f93fc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:49.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.665+0000 7f941bfff640 1 --2- 172.21.15.37:0/3718056905 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f93fc078b90 0x7f93fc07b0a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9404035000 tx=0x7f9404004ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:49.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.669+0000 7f94197fa640 1 -- 172.21.15.37:0/3718056905 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f940c020020 con 0x7f941c15bc70 2024-09-17T04:33:49.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.891+0000 7f9422cb4640 1 -- 172.21.15.37:0/3718056905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f941c0cfb30 con 0x7f941c15bc70 2024-09-17T04:33:49.891 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-09-17T04:33:49.892+0000 7f4732e82640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T04:33:49.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.893+0000 7f94197fa640 1 -- 172.21.15.37:0/3718056905 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f940c062d00 con 0x7f941c15bc70 2024-09-17T04:33:49.893 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T04:33:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.896+0000 7f9422cb4640 1 -- 172.21.15.37:0/3718056905 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f93fc078b90 msgr2=0x7f93fc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.896+0000 7f9422cb4640 1 --2- 172.21.15.37:0/3718056905 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f93fc078b90 0x7f93fc07b0a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9404035000 tx=0x7f9404004ab0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.896+0000 7f9422cb4640 1 -- 172.21.15.37:0/3718056905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f941c15bc70 msgr2=0x7f941c156260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.896+0000 7f9422cb4640 1 --2- 172.21.15.37:0/3718056905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f941c15bc70 0x7f941c156260 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f940c00b420 tx=0x7f940c00b8e0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.896+0000 7f9422cb4640 1 -- 172.21.15.37:0/3718056905 shutdown_connections 2024-09-17T04:33:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.896+0000 7f9422cb4640 1 --2- 172.21.15.37:0/3718056905 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f93fc078b90 0x7f93fc07b0a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.896+0000 7f9422cb4640 1 --2- 172.21.15.37:0/3718056905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f941c0b2f20 0x7f941c159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.896+0000 7f9422cb4640 1 --2- 172.21.15.37:0/3718056905 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f941c15d960 0x7f941c1e7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.896+0000 7f9422cb4640 1 --2- 172.21.15.37:0/3718056905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f941c15bc70 0x7f941c156260 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:49.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.896+0000 7f9422cb4640 1 -- 172.21.15.37:0/3718056905 >> 172.21.15.37:0/3718056905 conn(0x7f941c069120 msgr2=0x7f941c0b7260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:49.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.896+0000 7f9422cb4640 1 -- 172.21.15.37:0/3718056905 shutdown_connections 2024-09-17T04:33:49.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:49.896+0000 7f9422cb4640 1 -- 172.21.15.37:0/3718056905 wait complete. 2024-09-17T04:33:49.897 INFO:tasks.ceph.mds.b.smithi092.stderr: -3558> 2024-09-17T04:33:13.652+0000 7fc8e9dad640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-17T04:33:49.898 INFO:tasks.ceph.mds.b.smithi092.stderr: -2157> 2024-09-17T04:33:24.653+0000 7fc8e3da1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T04:33:49.908 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T04:33:49.953 INFO:tasks.ceph.mds.b.smithi092.stdout:starting mds.b at 2024-09-17T04:33:50.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.029+0000 7f55c17c3640 1 -- 172.21.15.37:0/3908000819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55a40053a0 msgr2=0x7f55a400ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:50.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.029+0000 7f55c17c3640 1 --2- 172.21.15.37:0/3908000819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55a40053a0 0x7f55a400ba20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f55b4001f00 tx=0x7f55a4005830 comp rx=0 tx=0).stop 2024-09-17T04:33:50.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.029+0000 7f55bb7fe640 1 -- 172.21.15.37:0/3908000819 reap_dead start 2024-09-17T04:33:50.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.029+0000 7f55c17c3640 1 -- 172.21.15.37:0/3908000819 shutdown_connections 2024-09-17T04:33:50.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.029+0000 7f55c17c3640 1 -- 172.21.15.37:0/3908000819 >> 172.21.15.37:0/3908000819 conn(0x7f55bc09d790 msgr2=0x7f55bc0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:50.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.029+0000 7f55c17c3640 1 -- 172.21.15.37:0/3908000819 shutdown_connections 2024-09-17T04:33:50.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.029+0000 7f55c17c3640 1 -- 172.21.15.37:0/3908000819 wait complete. 2024-09-17T04:33:50.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.031+0000 7f55c17c3640 1 Processor -- start 2024-09-17T04:33:50.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.031+0000 7f55c17c3640 1 -- start start 2024-09-17T04:33:50.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.032+0000 7f55c17c3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55a40053a0 0x7f55bc0a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:50.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.032+0000 7f55c17c3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55bc0cd3e0 0x7f55bc1f8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:50.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.032+0000 7f55c17c3640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55bc14f470 0x7f55bc152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:50.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.032+0000 7f55c17c3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc0b6a50 con 0x7f55a40053a0 2024-09-17T04:33:50.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.032+0000 7f55c17c3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc0b68d0 con 0x7f55bc0cd3e0 2024-09-17T04:33:50.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.032+0000 7f55c17c3640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f55bc0b1030 con 0x7f55bc14f470 2024-09-17T04:33:50.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.032+0000 7f55baffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55a40053a0 0x7f55bc0a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:50.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.032+0000 7f55baffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55a40053a0 0x7f55bc0a1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49392/0 (socket says 172.21.15.37:49392) 2024-09-17T04:33:50.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.032+0000 7f55baffd640 1 -- 172.21.15.37:0/1529293473 learned_addr learned my addr 172.21.15.37:0/1529293473 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:50.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.032+0000 7f55bb7fe640 1 --2- 172.21.15.37:0/1529293473 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55bc14f470 0x7f55bc152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:50.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.032+0000 7f55ba7fc640 1 --2- 172.21.15.37:0/1529293473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55bc0cd3e0 0x7f55bc1f8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:50.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.032+0000 7f55baffd640 1 -- 172.21.15.37:0/1529293473 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55bc14f470 msgr2=0x7f55bc152390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:50.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.032+0000 7f55baffd640 1 --2- 172.21.15.37:0/1529293473 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55bc14f470 0x7f55bc152390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:50.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.032+0000 7f55baffd640 1 -- 172.21.15.37:0/1529293473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55bc0cd3e0 msgr2=0x7f55bc1f8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:50.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.032+0000 7f55baffd640 1 --2- 172.21.15.37:0/1529293473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55bc0cd3e0 0x7f55bc1f8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:50.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.032+0000 7f55baffd640 1 -- 172.21.15.37:0/1529293473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55bc15f0a0 con 0x7f55a40053a0 2024-09-17T04:33:50.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.032+0000 7f55ba7fc640 1 --2- 172.21.15.37:0/1529293473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55bc0cd3e0 0x7f55bc1f8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:50.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.033+0000 7f55baffd640 1 --2- 172.21.15.37:0/1529293473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55a40053a0 0x7f55bc0a1dc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f55ac001220 tx=0x7f55ac0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:50.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.033+0000 7f55a37fe640 1 -- 172.21.15.37:0/1529293473 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac015070 con 0x7f55a40053a0 2024-09-17T04:33:50.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.033+0000 7f55a37fe640 1 -- 172.21.15.37:0/1529293473 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55ac011840 con 0x7f55a40053a0 2024-09-17T04:33:50.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.033+0000 7f55a37fe640 1 -- 172.21.15.37:0/1529293473 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac011c90 con 0x7f55a40053a0 2024-09-17T04:33:50.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.033+0000 7f55c17c3640 1 -- 172.21.15.37:0/1529293473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55bc1f9d30 con 0x7f55a40053a0 2024-09-17T04:33:50.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.033+0000 7f55c17c3640 1 -- 172.21.15.37:0/1529293473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55bc1f9770 con 0x7f55a40053a0 2024-09-17T04:33:50.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.034+0000 7f55c17c3640 1 -- 172.21.15.37:0/1529293473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55bc0d2f70 con 0x7f55a40053a0 2024-09-17T04:33:50.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.035+0000 7f55a37fe640 1 -- 172.21.15.37:0/1529293473 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f55ac018070 con 0x7f55a40053a0 2024-09-17T04:33:50.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.035+0000 7f55a37fe640 1 --2- 172.21.15.37:0/1529293473 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f55a8078bc0 0x7f55a807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:50.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.035+0000 7f55a37fe640 1 -- 172.21.15.37:0/1529293473 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5795+0+0 (secure 0 0 0) 0x7f55ac09bad0 con 0x7f55a40053a0 2024-09-17T04:33:50.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.035+0000 7f55ba7fc640 1 --2- 172.21.15.37:0/1529293473 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f55a8078bc0 0x7f55a807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:50.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.036+0000 7f55ba7fc640 1 --2- 172.21.15.37:0/1529293473 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f55a8078bc0 0x7f55a807b0d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f55bc0b6c80 tx=0x7f55b0005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:50.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.040+0000 7f55a37fe640 1 -- 172.21.15.37:0/1529293473 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f55ac0637e0 con 0x7f55a40053a0 2024-09-17T04:33:50.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.264+0000 7f55c17c3640 1 -- 172.21.15.37:0/1529293473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f55bc161220 con 0x7f55a40053a0 2024-09-17T04:33:50.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.895+0000 7f55a37fe640 1 -- 172.21.15.37:0/1529293473 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f55ac068210 con 0x7f55a40053a0 2024-09-17T04:33:50.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.898+0000 7f55c17c3640 1 -- 172.21.15.37:0/1529293473 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f55a8078bc0 msgr2=0x7f55a807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:50.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.898+0000 7f55c17c3640 1 --2- 172.21.15.37:0/1529293473 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f55a8078bc0 0x7f55a807b0d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f55bc0b6c80 tx=0x7f55b0005f90 comp rx=0 tx=0).stop 2024-09-17T04:33:50.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.898+0000 7f55c17c3640 1 -- 172.21.15.37:0/1529293473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55a40053a0 msgr2=0x7f55bc0a1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:50.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.898+0000 7f55c17c3640 1 --2- 172.21.15.37:0/1529293473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55a40053a0 0x7f55bc0a1dc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f55ac001220 tx=0x7f55ac0039c0 comp rx=0 tx=0).stop 2024-09-17T04:33:50.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.898+0000 7f55c17c3640 1 -- 172.21.15.37:0/1529293473 shutdown_connections 2024-09-17T04:33:50.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.898+0000 7f55c17c3640 1 --2- 172.21.15.37:0/1529293473 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f55a8078bc0 0x7f55a807b0d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:50.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.898+0000 7f55c17c3640 1 --2- 172.21.15.37:0/1529293473 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55bc14f470 0x7f55bc152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:50.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.898+0000 7f55c17c3640 1 --2- 172.21.15.37:0/1529293473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55bc0cd3e0 0x7f55bc1f8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:50.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.898+0000 7f55c17c3640 1 --2- 172.21.15.37:0/1529293473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55a40053a0 0x7f55bc0a1dc0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:50.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.898+0000 7f55c17c3640 1 -- 172.21.15.37:0/1529293473 >> 172.21.15.37:0/1529293473 conn(0x7f55bc09d790 msgr2=0x7f55bc162920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:50.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.898+0000 7f55c17c3640 1 -- 172.21.15.37:0/1529293473 shutdown_connections 2024-09-17T04:33:50.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:50.898+0000 7f55c17c3640 1 -- 172.21.15.37:0/1529293473 wait complete. 2024-09-17T04:33:50.912 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T04:33:51.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.042+0000 7f25f27ba640 1 -- 172.21.15.37:0/3747118953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25ec0d1af0 msgr2=0x7f25ec0b9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:51.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.042+0000 7f25f27ba640 1 --2- 172.21.15.37:0/3747118953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25ec0d1af0 0x7f25ec0b9630 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f25dc009820 tx=0x7f25dc02a480 comp rx=0 tx=0).stop 2024-09-17T04:33:51.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.042+0000 7f25f27ba640 1 -- 172.21.15.37:0/3747118953 shutdown_connections 2024-09-17T04:33:51.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.042+0000 7f25f27ba640 1 --2- 172.21.15.37:0/3747118953 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f25ec0d2c00 0x7f25ec0d3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:51.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.042+0000 7f25f27ba640 1 --2- 172.21.15.37:0/3747118953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25ec0d1af0 0x7f25ec0b9630 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:51.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.042+0000 7f25f27ba640 1 -- 172.21.15.37:0/3747118953 >> 172.21.15.37:0/3747118953 conn(0x7f25ec09d6f0 msgr2=0x7f25ec0d4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:51.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.042+0000 7f25f27ba640 1 -- 172.21.15.37:0/3747118953 shutdown_connections 2024-09-17T04:33:51.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.042+0000 7f25f27ba640 1 -- 172.21.15.37:0/3747118953 wait complete. 2024-09-17T04:33:51.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.044+0000 7f25f27ba640 1 Processor -- start 2024-09-17T04:33:51.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.044+0000 7f25f27ba640 1 -- start start 2024-09-17T04:33:51.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.046+0000 7f25f27ba640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25ec0b3b10 0x7f25ec1d7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:51.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.046+0000 7f25f27ba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25ec0d1af0 0x7f25ec0b82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:51.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.046+0000 7f25f27ba640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f25ec0d2c00 0x7f25ec0b87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:51.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.046+0000 7f25f27ba640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec0d11e0 con 0x7f25ec0d1af0 2024-09-17T04:33:51.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.046+0000 7f25f27ba640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec0d1060 con 0x7f25ec0b3b10 2024-09-17T04:33:51.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.046+0000 7f25f27ba640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f25ec0d1360 con 0x7f25ec0d2c00 2024-09-17T04:33:51.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.046+0000 7f25eb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25ec0d1af0 0x7f25ec0b82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:51.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.046+0000 7f25eb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25ec0d1af0 0x7f25ec0b82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49396/0 (socket says 172.21.15.37:49396) 2024-09-17T04:33:51.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.046+0000 7f25eb7fe640 1 -- 172.21.15.37:0/828603645 learned_addr learned my addr 172.21.15.37:0/828603645 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:51.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.046+0000 7f25ebfff640 1 --2- 172.21.15.37:0/828603645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25ec0b3b10 0x7f25ec1d7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:51.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.046+0000 7f25f0d57640 1 --2- 172.21.15.37:0/828603645 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f25ec0d2c00 0x7f25ec0b87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:51.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.046+0000 7f25eb7fe640 1 -- 172.21.15.37:0/828603645 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f25ec0d2c00 msgr2=0x7f25ec0b87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:51.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.046+0000 7f25eb7fe640 1 --2- 172.21.15.37:0/828603645 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f25ec0d2c00 0x7f25ec0b87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:51.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.046+0000 7f25eb7fe640 1 -- 172.21.15.37:0/828603645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25ec0b3b10 msgr2=0x7f25ec1d7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:51.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.046+0000 7f25eb7fe640 1 --2- 172.21.15.37:0/828603645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25ec0b3b10 0x7f25ec1d7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:51.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.046+0000 7f25eb7fe640 1 -- 172.21.15.37:0/828603645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ec0c8710 con 0x7f25ec0d1af0 2024-09-17T04:33:51.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.046+0000 7f25eb7fe640 1 --2- 172.21.15.37:0/828603645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25ec0d1af0 0x7f25ec0b82a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f25e000e9c0 tx=0x7f25e000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:51.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.047+0000 7f25e8ff9640 1 -- 172.21.15.37:0/828603645 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e000cd80 con 0x7f25ec0d1af0 2024-09-17T04:33:51.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.047+0000 7f25e8ff9640 1 -- 172.21.15.37:0/828603645 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25e0004020 con 0x7f25ec0d1af0 2024-09-17T04:33:51.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.047+0000 7f25f27ba640 1 -- 172.21.15.37:0/828603645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25ec0c89f0 con 0x7f25ec0d1af0 2024-09-17T04:33:51.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.047+0000 7f25e8ff9640 1 -- 172.21.15.37:0/828603645 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e0004450 con 0x7f25ec0d1af0 2024-09-17T04:33:51.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.047+0000 7f25f27ba640 1 -- 172.21.15.37:0/828603645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25ec0c9ba0 con 0x7f25ec0d1af0 2024-09-17T04:33:51.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.048+0000 7f25f27ba640 1 -- 172.21.15.37:0/828603645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25ec0c8480 con 0x7f25ec0d1af0 2024-09-17T04:33:51.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.049+0000 7f25e8ff9640 1 -- 172.21.15.37:0/828603645 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f25e00035f0 con 0x7f25ec0d1af0 2024-09-17T04:33:51.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.049+0000 7f25e8ff9640 1 --2- 172.21.15.37:0/828603645 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f25d8078bc0 0x7f25d807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:51.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.049+0000 7f25e8ff9640 1 -- 172.21.15.37:0/828603645 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5795+0+0 (secure 0 0 0) 0x7f25e009b790 con 0x7f25ec0d1af0 2024-09-17T04:33:51.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.050+0000 7f25ebfff640 1 --2- 172.21.15.37:0/828603645 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f25d8078bc0 0x7f25d807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:51.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.050+0000 7f25ebfff640 1 --2- 172.21.15.37:0/828603645 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f25d8078bc0 0x7f25d807b0d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f25dc034000 tx=0x7f25dc004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:51.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.053+0000 7f25e8ff9640 1 -- 172.21.15.37:0/828603645 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f25e00635b0 con 0x7f25ec0d1af0 2024-09-17T04:33:51.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:51.238+0000 7f25f27ba640 1 -- 172.21.15.37:0/828603645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f25ec0b9630 con 0x7f25ec0d1af0 2024-09-17T04:33:52.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.008+0000 7f25e8ff9640 1 -- 172.21.15.37:0/828603645 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) ==== 181+0+0 (secure 0 0 0) 0x7f25e0067fe0 con 0x7f25ec0d1af0 2024-09-17T04:33:52.008 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-09-17T04:33:52.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.011+0000 7f25f27ba640 1 -- 172.21.15.37:0/828603645 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f25d8078bc0 msgr2=0x7f25d807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:52.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.011+0000 7f25f27ba640 1 --2- 172.21.15.37:0/828603645 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f25d8078bc0 0x7f25d807b0d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f25dc034000 tx=0x7f25dc004730 comp rx=0 tx=0).stop 2024-09-17T04:33:52.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.011+0000 7f25f27ba640 1 -- 172.21.15.37:0/828603645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25ec0d1af0 msgr2=0x7f25ec0b82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:52.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.011+0000 7f25f27ba640 1 --2- 172.21.15.37:0/828603645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25ec0d1af0 0x7f25ec0b82a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f25e000e9c0 tx=0x7f25e000ee80 comp rx=0 tx=0).stop 2024-09-17T04:33:52.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.013+0000 7f25f27ba640 1 -- 172.21.15.37:0/828603645 shutdown_connections 2024-09-17T04:33:52.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.013+0000 7f25f27ba640 1 --2- 172.21.15.37:0/828603645 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f25d8078bc0 0x7f25d807b0d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:52.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.013+0000 7f25f27ba640 1 --2- 172.21.15.37:0/828603645 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f25ec0d2c00 0x7f25ec0b87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:52.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.013+0000 7f25f27ba640 1 --2- 172.21.15.37:0/828603645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25ec0d1af0 0x7f25ec0b82a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:52.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.013+0000 7f25f27ba640 1 --2- 172.21.15.37:0/828603645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25ec0b3b10 0x7f25ec1d7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:52.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.013+0000 7f25f27ba640 1 -- 172.21.15.37:0/828603645 >> 172.21.15.37:0/828603645 conn(0x7f25ec09d6f0 msgr2=0x7f25ec0b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:52.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.013+0000 7f25f27ba640 1 -- 172.21.15.37:0/828603645 shutdown_connections 2024-09-17T04:33:52.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.013+0000 7f25f27ba640 1 -- 172.21.15.37:0/828603645 wait complete. 2024-09-17T04:33:52.029 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T04:33:52.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.151+0000 7fa9bc924640 1 -- 172.21.15.37:0/1337268206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9b806f280 msgr2=0x7fa9b80b42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:52.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.151+0000 7fa9bc924640 1 --2- 172.21.15.37:0/1337268206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9b806f280 0x7fa9b80b42c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8072710 tx=0x7fa9b002a8b0 comp rx=0 tx=0).stop 2024-09-17T04:33:52.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.152+0000 7fa9bc924640 1 -- 172.21.15.37:0/1337268206 shutdown_connections 2024-09-17T04:33:52.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.152+0000 7fa9bc924640 1 --2- 172.21.15.37:0/1337268206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9b806f280 0x7fa9b80b42c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:52.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.152+0000 7fa9bc924640 1 --2- 172.21.15.37:0/1337268206 >> v2:172.21.15.92:3300/0 conn(0x7fa9b80d0620 0x7fa9b806ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:52.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.152+0000 7fa9bc924640 1 -- 172.21.15.37:0/1337268206 >> 172.21.15.37:0/1337268206 conn(0x7fa9b806cd60 msgr2=0x7fa9b806d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:52.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.152+0000 7fa9bc924640 1 -- 172.21.15.37:0/1337268206 shutdown_connections 2024-09-17T04:33:52.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.152+0000 7fa9bc924640 1 -- 172.21.15.37:0/1337268206 wait complete. 2024-09-17T04:33:52.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.154+0000 7fa9bc924640 1 Processor -- start 2024-09-17T04:33:52.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.154+0000 7fa9bc924640 1 -- start start 2024-09-17T04:33:52.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.155+0000 7fa9bc924640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa9b806f280 0x7fa9b8072c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:52.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.155+0000 7fa9bc924640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9b80d0620 0x7fa9b80731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:52.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.155+0000 7fa9bc924640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9b81600f0 0x7fa9b80736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:52.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.155+0000 7fa9bc924640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b80b4b90 con 0x7fa9b81600f0 2024-09-17T04:33:52.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.155+0000 7fa9bc924640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b80b4a10 con 0x7fa9b80d0620 2024-09-17T04:33:52.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.155+0000 7fa9bc924640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fa9b80b4d10 con 0x7fa9b806f280 2024-09-17T04:33:52.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.155+0000 7fa9b6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9b81600f0 0x7fa9b80736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:52.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.155+0000 7fa9b6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9b81600f0 0x7fa9b80736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49416/0 (socket says 172.21.15.37:49416) 2024-09-17T04:33:52.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.155+0000 7fa9b6d9d640 1 -- 172.21.15.37:0/1788714035 learned_addr learned my addr 172.21.15.37:0/1788714035 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:52.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.155+0000 7fa9affff640 1 --2- 172.21.15.37:0/1788714035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9b80d0620 0x7fa9b80731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:52.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.155+0000 7fa9b659c640 1 --2- 172.21.15.37:0/1788714035 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa9b806f280 0x7fa9b8072c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:52.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.156+0000 7fa9b6d9d640 1 -- 172.21.15.37:0/1788714035 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa9b806f280 msgr2=0x7fa9b8072c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:52.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.156+0000 7fa9b6d9d640 1 --2- 172.21.15.37:0/1788714035 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa9b806f280 0x7fa9b8072c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:52.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.156+0000 7fa9b6d9d640 1 -- 172.21.15.37:0/1788714035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9b80d0620 msgr2=0x7fa9b80731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:52.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.156+0000 7fa9b6d9d640 1 --2- 172.21.15.37:0/1788714035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9b80d0620 0x7fa9b80731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:52.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.156+0000 7fa9b6d9d640 1 -- 172.21.15.37:0/1788714035 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b80cc0d0 con 0x7fa9b81600f0 2024-09-17T04:33:52.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.156+0000 7fa9b6d9d640 1 --2- 172.21.15.37:0/1788714035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9b81600f0 0x7fa9b80736e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0009ca0 tx=0x7fa9b0003990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:52.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.156+0000 7fa9aeffd640 1 -- 172.21.15.37:0/1788714035 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0031990 con 0x7fa9b81600f0 2024-09-17T04:33:52.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.156+0000 7fa9aeffd640 1 -- 172.21.15.37:0/1788714035 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9b0031c20 con 0x7fa9b81600f0 2024-09-17T04:33:52.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.156+0000 7fa9bc924640 1 -- 172.21.15.37:0/1788714035 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b8146d90 con 0x7fa9b81600f0 2024-09-17T04:33:52.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.156+0000 7fa9aeffd640 1 -- 172.21.15.37:0/1788714035 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0033070 con 0x7fa9b81600f0 2024-09-17T04:33:52.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.156+0000 7fa9bc924640 1 -- 172.21.15.37:0/1788714035 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b81488f0 con 0x7fa9b81600f0 2024-09-17T04:33:52.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.157+0000 7fa9bc924640 1 -- 172.21.15.37:0/1788714035 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9b80b5250 con 0x7fa9b81600f0 2024-09-17T04:33:52.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.158+0000 7fa9aeffd640 1 -- 172.21.15.37:0/1788714035 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa9b0033210 con 0x7fa9b81600f0 2024-09-17T04:33:52.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.158+0000 7fa9aeffd640 1 --2- 172.21.15.37:0/1788714035 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa998078b90 0x7fa99807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:52.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.158+0000 7fa9aeffd640 1 -- 172.21.15.37:0/1788714035 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5357+0+0 (secure 0 0 0) 0x7fa9b00bc140 con 0x7fa9b81600f0 2024-09-17T04:33:52.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.159+0000 7fa9b659c640 1 --2- 172.21.15.37:0/1788714035 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa998078b90 0x7fa99807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:52.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.159+0000 7fa9b659c640 1 --2- 172.21.15.37:0/1788714035 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa998078b90 0x7fa99807b0a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8001720 tx=0x7fa9a8007e90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:52.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.163+0000 7fa9aeffd640 1 -- 172.21.15.37:0/1788714035 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa9b003f030 con 0x7fa9b81600f0 2024-09-17T04:33:52.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:52.356+0000 7fa9bc924640 1 -- 172.21.15.37:0/1788714035 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa9b8156870 con 0x7fa9b81600f0 2024-09-17T04:33:53.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.019+0000 7fa9aeffd640 1 -- 172.21.15.37:0/1788714035 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) ==== 169+0+0 (secure 0 0 0) 0x7fa9b0083ff0 con 0x7fa9b81600f0 2024-09-17T04:33:53.019 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-09-17T04:33:53.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.023+0000 7fa9bc924640 1 -- 172.21.15.37:0/1788714035 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa998078b90 msgr2=0x7fa99807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:53.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.023+0000 7fa9bc924640 1 --2- 172.21.15.37:0/1788714035 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa998078b90 0x7fa99807b0a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8001720 tx=0x7fa9a8007e90 comp rx=0 tx=0).stop 2024-09-17T04:33:53.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.023+0000 7fa9bc924640 1 -- 172.21.15.37:0/1788714035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9b81600f0 msgr2=0x7fa9b80736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:53.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.023+0000 7fa9bc924640 1 --2- 172.21.15.37:0/1788714035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9b81600f0 0x7fa9b80736e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0009ca0 tx=0x7fa9b0003990 comp rx=0 tx=0).stop 2024-09-17T04:33:53.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.023+0000 7fa9bc924640 1 -- 172.21.15.37:0/1788714035 shutdown_connections 2024-09-17T04:33:53.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.023+0000 7fa9bc924640 1 --2- 172.21.15.37:0/1788714035 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa998078b90 0x7fa99807b0a0 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8001720 tx=0x7fa9a8007e90 comp rx=0 tx=0).stop 2024-09-17T04:33:53.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.023+0000 7fa9bc924640 1 --2- 172.21.15.37:0/1788714035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9b81600f0 0x7fa9b80736e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:53.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.023+0000 7fa9bc924640 1 --2- 172.21.15.37:0/1788714035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9b80d0620 0x7fa9b80731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:53.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.023+0000 7fa9bc924640 1 --2- 172.21.15.37:0/1788714035 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa9b806f280 0x7fa9b8072c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:53.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.023+0000 7fa9bc924640 1 -- 172.21.15.37:0/1788714035 >> 172.21.15.37:0/1788714035 conn(0x7fa9b806cd60 msgr2=0x7fa9b815b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:53.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.024+0000 7fa9bc924640 1 -- 172.21.15.37:0/1788714035 shutdown_connections 2024-09-17T04:33:53.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.024+0000 7fa9bc924640 1 -- 172.21.15.37:0/1788714035 wait complete. 2024-09-17T04:33:53.040 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T04:33:53.040 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T04:33:53.040 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T04:33:53.040 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:33:53.041 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T04:33:53.041 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T04:33:53.041 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T04:33:53.041 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:33:53.041 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T04:33:53.041 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T04:33:53.041 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T04:33:53.042 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:33:53.042 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T04:33:53.042 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T04:33:53.042 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T04:33:53.042 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:33:53.052 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T04:33:53.052 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T04:33:53.053 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T04:33:53.053 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T04:33:53.055 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T04:33:53.055 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T04:33:53.058 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T04:33:53.058 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T04:33:53.094 INFO:tasks.ceph.mds.a:Started 2024-09-17T04:33:53.095 DEBUG:teuthology.parallel:result is None 2024-09-17T04:33:53.095 INFO:tasks.ceph.mds.c:Started 2024-09-17T04:33:53.096 DEBUG:teuthology.parallel:result is None 2024-09-17T04:33:53.099 INFO:tasks.ceph.mds.d:Started 2024-09-17T04:33:53.100 DEBUG:teuthology.parallel:result is None 2024-09-17T04:33:53.101 INFO:tasks.ceph.mds.b:Started 2024-09-17T04:33:53.101 DEBUG:teuthology.parallel:result is None 2024-09-17T04:33:53.101 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T04:33:53.187 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-09-17T04:33:53.187 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-09-17T04:33:53.202 INFO:tasks.ceph.mds.b.smithi092.stdout:starting mds.b at 2024-09-17T04:33:53.202 INFO:tasks.ceph.mds.d.smithi092.stdout:starting mds.d at 2024-09-17T04:33:53.203 INFO:tasks.ceph.mds.d.smithi092.stderr:2024-09-17T04:33:53.203+0000 7efcf0975640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-17T04:33:53.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.229+0000 7f17a8e8b640 1 -- 172.21.15.37:0/668324932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f178c003b30 msgr2=0x7f178c003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:53.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.229+0000 7f17a8e8b640 1 --2- 172.21.15.37:0/668324932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f178c003b30 0x7f178c003fa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1794009b50 tx=0x7f178c005830 comp rx=0 tx=0).stop 2024-09-17T04:33:53.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.229+0000 7f17a8e8b640 1 -- 172.21.15.37:0/668324932 shutdown_connections 2024-09-17T04:33:53.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.229+0000 7f17a8e8b640 1 --2- 172.21.15.37:0/668324932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f178c003b30 0x7f178c003fa0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:53.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.229+0000 7f17a8e8b640 1 -- 172.21.15.37:0/668324932 >> 172.21.15.37:0/668324932 conn(0x7f17a4069120 msgr2=0x7f17a4069530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:53.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.229+0000 7f17a8e8b640 1 -- 172.21.15.37:0/668324932 shutdown_connections 2024-09-17T04:33:53.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.229+0000 7f17a8e8b640 1 -- 172.21.15.37:0/668324932 wait complete. 2024-09-17T04:33:53.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.231+0000 7f17a8e8b640 1 Processor -- start 2024-09-17T04:33:53.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.231+0000 7f17a8e8b640 1 -- start start 2024-09-17T04:33:53.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.232+0000 7f17a8e8b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f178c003b30 0x7f17a4142dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:53.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.232+0000 7f17a8e8b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f17a4159bc0 0x7f17a4159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:53.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.232+0000 7f17a8e8b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17a415b6a0 0x7f17a415bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:53.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.232+0000 7f17a8e8b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f17a4157270 con 0x7f17a415b6a0 2024-09-17T04:33:53.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.232+0000 7f17a8e8b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f17a415dd70 con 0x7f178c003b30 2024-09-17T04:33:53.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.232+0000 7f17a8e8b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f17a41573f0 con 0x7f17a4159bc0 2024-09-17T04:33:53.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.232+0000 7f17a2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17a415b6a0 0x7f17a415bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:53.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.233+0000 7f17a2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17a415b6a0 0x7f17a415bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49470/0 (socket says 172.21.15.37:49470) 2024-09-17T04:33:53.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.233+0000 7f17a2d9d640 1 -- 172.21.15.37:0/3762315090 learned_addr learned my addr 172.21.15.37:0/3762315090 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:53.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.233+0000 7f17a1d9b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f17a4159bc0 0x7f17a4159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:53.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.233+0000 7f17a259c640 1 --2- 172.21.15.37:0/3762315090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f178c003b30 0x7f17a4142dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:53.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.233+0000 7f17a2d9d640 1 -- 172.21.15.37:0/3762315090 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f17a4159bc0 msgr2=0x7f17a4159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:53.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.233+0000 7f17a2d9d640 1 --2- 172.21.15.37:0/3762315090 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f17a4159bc0 0x7f17a4159fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:53.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.233+0000 7f17a2d9d640 1 -- 172.21.15.37:0/3762315090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f178c003b30 msgr2=0x7f17a4142dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:53.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.233+0000 7f17a2d9d640 1 --2- 172.21.15.37:0/3762315090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f178c003b30 0x7f17a4142dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:53.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.233+0000 7f17a2d9d640 1 -- 172.21.15.37:0/3762315090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a415bff0 con 0x7f17a415b6a0 2024-09-17T04:33:53.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.233+0000 7f17a1d9b640 1 --2- 172.21.15.37:0/3762315090 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f17a4159bc0 0x7f17a4159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:33:53.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.233+0000 7f17a2d9d640 1 --2- 172.21.15.37:0/3762315090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17a415b6a0 0x7f17a415bab0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f17a40b1c50 tx=0x7f179c006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:53.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.233+0000 7f1792ffd640 1 -- 172.21.15.37:0/3762315090 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c0119c0 con 0x7f17a415b6a0 2024-09-17T04:33:53.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.233+0000 7f1792ffd640 1 -- 172.21.15.37:0/3762315090 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f179c011b80 con 0x7f17a415b6a0 2024-09-17T04:33:53.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.233+0000 7f17a8e8b640 1 -- 172.21.15.37:0/3762315090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a41f3590 con 0x7f17a415b6a0 2024-09-17T04:33:53.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.233+0000 7f1792ffd640 1 -- 172.21.15.37:0/3762315090 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c0103c0 con 0x7f17a415b6a0 2024-09-17T04:33:53.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.233+0000 7f17a8e8b640 1 -- 172.21.15.37:0/3762315090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a41f3b50 con 0x7f17a415b6a0 2024-09-17T04:33:53.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.234+0000 7f17a8e8b640 1 -- 172.21.15.37:0/3762315090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17a40cf1e0 con 0x7f17a415b6a0 2024-09-17T04:33:53.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.235+0000 7f1792ffd640 1 -- 172.21.15.37:0/3762315090 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f179c010560 con 0x7f17a415b6a0 2024-09-17T04:33:53.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.236+0000 7f1792ffd640 1 --2- 172.21.15.37:0/3762315090 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1784078b90 0x7f178407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:53.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.236+0000 7f1792ffd640 1 -- 172.21.15.37:0/3762315090 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4943+0+0 (secure 0 0 0) 0x7f179c09b170 con 0x7f17a415b6a0 2024-09-17T04:33:53.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.236+0000 7f17a259c640 1 --2- 172.21.15.37:0/3762315090 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1784078b90 0x7f178407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:53.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.237+0000 7f17a259c640 1 --2- 172.21.15.37:0/3762315090 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1784078b90 0x7f178407b0a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1794035000 tx=0x7f17940046c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:53.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.240+0000 7f1792ffd640 1 -- 172.21.15.37:0/3762315090 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f17a415bff0 con 0x7f17a415b6a0 2024-09-17T04:33:53.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:53.438+0000 7f17a8e8b640 1 -- 172.21.15.37:0/3762315090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f17a40008d0 con 0x7f17a415b6a0 2024-09-17T04:33:54.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.030+0000 7f1792ffd640 1 -- 172.21.15.37:0/3762315090 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) ==== 97+0+0 (secure 0 0 0) 0x7f17a40cfb30 con 0x7f17a415b6a0 2024-09-17T04:33:54.030 INFO:teuthology.orchestra.run.smithi037.stderr: removed all blocklist entries 2024-09-17T04:33:54.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.033+0000 7f17a8e8b640 1 -- 172.21.15.37:0/3762315090 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1784078b90 msgr2=0x7f178407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:54.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.033+0000 7f17a8e8b640 1 --2- 172.21.15.37:0/3762315090 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1784078b90 0x7f178407b0a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1794035000 tx=0x7f17940046c0 comp rx=0 tx=0).stop 2024-09-17T04:33:54.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.033+0000 7f17a8e8b640 1 -- 172.21.15.37:0/3762315090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17a415b6a0 msgr2=0x7f17a415bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:54.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.033+0000 7f17a8e8b640 1 --2- 172.21.15.37:0/3762315090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17a415b6a0 0x7f17a415bab0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f17a40b1c50 tx=0x7f179c006430 comp rx=0 tx=0).stop 2024-09-17T04:33:54.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.033+0000 7f17a8e8b640 1 -- 172.21.15.37:0/3762315090 shutdown_connections 2024-09-17T04:33:54.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.033+0000 7f17a8e8b640 1 --2- 172.21.15.37:0/3762315090 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1784078b90 0x7f178407b0a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:54.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.033+0000 7f17a8e8b640 1 --2- 172.21.15.37:0/3762315090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17a415b6a0 0x7f17a415bab0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:54.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.033+0000 7f17a8e8b640 1 --2- 172.21.15.37:0/3762315090 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f17a4159bc0 0x7f17a4159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:54.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.033+0000 7f17a8e8b640 1 --2- 172.21.15.37:0/3762315090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f178c003b30 0x7f17a4142dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:54.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.033+0000 7f17a8e8b640 1 -- 172.21.15.37:0/3762315090 >> 172.21.15.37:0/3762315090 conn(0x7f17a4069120 msgr2=0x7f17a415ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:54.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.033+0000 7f17a8e8b640 1 -- 172.21.15.37:0/3762315090 shutdown_connections 2024-09-17T04:33:54.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.033+0000 7f17a8e8b640 1 -- 172.21.15.37:0/3762315090 wait complete. 2024-09-17T04:33:54.048 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T04:33:54.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.170+0000 7f9cf83c3640 1 -- 172.21.15.37:0/3509793351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf00b1f80 msgr2=0x7f9cf006cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:54.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.170+0000 7f9cf83c3640 1 --2- 172.21.15.37:0/3509793351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf00b1f80 0x7f9cf006cd60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0009830 tx=0x7f9ce002a480 comp rx=0 tx=0).stop 2024-09-17T04:33:54.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.170+0000 7f9cf83c3640 1 -- 172.21.15.37:0/3509793351 shutdown_connections 2024-09-17T04:33:54.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.170+0000 7f9cf83c3640 1 --2- 172.21.15.37:0/3509793351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf00b1f80 0x7f9cf006cd60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:54.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.170+0000 7f9cf83c3640 1 --2- 172.21.15.37:0/3509793351 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9cf015c730 0x7f9cf00ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:54.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.170+0000 7f9cf83c3640 1 -- 172.21.15.37:0/3509793351 >> 172.21.15.37:0/3509793351 conn(0x7f9cf00d1300 msgr2=0x7f9cf00d1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:54.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.170+0000 7f9cf83c3640 1 -- 172.21.15.37:0/3509793351 shutdown_connections 2024-09-17T04:33:54.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.170+0000 7f9cf83c3640 1 -- 172.21.15.37:0/3509793351 wait complete. 2024-09-17T04:33:54.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.172+0000 7f9cf83c3640 1 Processor -- start 2024-09-17T04:33:54.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.172+0000 7f9cf83c3640 1 -- start start 2024-09-17T04:33:54.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.173+0000 7f9cf83c3640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9cf00b1f80 0x7f9cf01f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:54.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.173+0000 7f9cf83c3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cf00b2390 0x7f9cf0158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:54.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.173+0000 7f9cf83c3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf015c730 0x7f9cf0160a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:54.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.173+0000 7f9cf83c3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf00b4860 con 0x7f9cf015c730 2024-09-17T04:33:54.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.173+0000 7f9cf83c3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf015cd50 con 0x7f9cf00b2390 2024-09-17T04:33:54.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.173+0000 7f9cf83c3640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f9cf00b49e0 con 0x7f9cf00b1f80 2024-09-17T04:33:54.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.174+0000 7f9cf6960640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf015c730 0x7f9cf0160a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:54.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.174+0000 7f9cf6960640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf015c730 0x7f9cf0160a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49484/0 (socket says 172.21.15.37:49484) 2024-09-17T04:33:54.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.174+0000 7f9cf6960640 1 -- 172.21.15.37:0/3288018852 learned_addr learned my addr 172.21.15.37:0/3288018852 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:54.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.174+0000 7f9cf615f640 1 --2- 172.21.15.37:0/3288018852 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9cf00b1f80 0x7f9cf01f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:54.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.174+0000 7f9cf595e640 1 --2- 172.21.15.37:0/3288018852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cf00b2390 0x7f9cf0158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:54.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.174+0000 7f9cf6960640 1 -- 172.21.15.37:0/3288018852 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9cf00b1f80 msgr2=0x7f9cf01f4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:54.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.174+0000 7f9cf6960640 1 --2- 172.21.15.37:0/3288018852 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9cf00b1f80 0x7f9cf01f4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:54.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.174+0000 7f9cf6960640 1 -- 172.21.15.37:0/3288018852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cf00b2390 msgr2=0x7f9cf0158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:54.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.174+0000 7f9cf6960640 1 --2- 172.21.15.37:0/3288018852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cf00b2390 0x7f9cf0158b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:54.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.174+0000 7f9cf6960640 1 -- 172.21.15.37:0/3288018852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf015d750 con 0x7f9cf015c730 2024-09-17T04:33:54.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.174+0000 7f9cf6960640 1 --2- 172.21.15.37:0/3288018852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf015c730 0x7f9cf0160a20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9ce800c6c0 tx=0x7f9ce800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:54.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.174+0000 7f9ce6ffd640 1 -- 172.21.15.37:0/3288018852 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8017070 con 0x7f9cf015c730 2024-09-17T04:33:54.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.174+0000 7f9ce6ffd640 1 -- 172.21.15.37:0/3288018852 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ce8003c60 con 0x7f9cf015c730 2024-09-17T04:33:54.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.174+0000 7f9cf83c3640 1 -- 172.21.15.37:0/3288018852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cf015f980 con 0x7f9cf015c730 2024-09-17T04:33:54.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.174+0000 7f9ce6ffd640 1 -- 172.21.15.37:0/3288018852 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce80052d0 con 0x7f9cf015c730 2024-09-17T04:33:54.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.174+0000 7f9cf83c3640 1 -- 172.21.15.37:0/3288018852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cf015ff40 con 0x7f9cf015c730 2024-09-17T04:33:54.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.175+0000 7f9cf83c3640 1 -- 172.21.15.37:0/3288018852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cf006d410 con 0x7f9cf015c730 2024-09-17T04:33:54.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.176+0000 7f9ce6ffd640 1 -- 172.21.15.37:0/3288018852 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9ce801a070 con 0x7f9cf015c730 2024-09-17T04:33:54.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.177+0000 7f9ce6ffd640 1 --2- 172.21.15.37:0/3288018852 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9cd4078bc0 0x7f9cd407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:54.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.177+0000 7f9ce6ffd640 1 -- 172.21.15.37:0/3288018852 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f9ce8099eb0 con 0x7f9cf015c730 2024-09-17T04:33:54.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.177+0000 7f9cf615f640 1 --2- 172.21.15.37:0/3288018852 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9cd4078bc0 0x7f9cd407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:54.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.178+0000 7f9cf615f640 1 --2- 172.21.15.37:0/3288018852 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9cd4078bc0 0x7f9cd407b0d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9cec00a490 tx=0x7f9cec00a420 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:54.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.182+0000 7f9ce6ffd640 1 -- 172.21.15.37:0/3288018852 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9ce8062070 con 0x7f9cf015c730 2024-09-17T04:33:54.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.443+0000 7f9cf83c3640 1 -- 172.21.15.37:0/3288018852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f9cf00ce5a0 con 0x7f9cf015c730 2024-09-17T04:33:54.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.443+0000 7f9ce6ffd640 1 -- 172.21.15.37:0/3288018852 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) ==== 80+0+6645 (secure 0 0 0) 0x7f9ce8066aa0 con 0x7f9cf015c730 2024-09-17T04:33:54.443 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:54.443 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T04:33:54.443 INFO:teuthology.orchestra.run.smithi037.stdout: "auth_dump": [ 2024-09-17T04:33:54.443 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T04:33:54.443 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.a", 2024-09-17T04:33:54.444 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBQBulmh1PhMhAALoHZDSZgGMyaEd8e4M+2KA==", 2024-09-17T04:33:54.444 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-17T04:33:54.444 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-17T04:33:54.444 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-17T04:33:54.444 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-17T04:33:54.444 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-17T04:33:54.444 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:54.444 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:54.444 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T04:33:54.444 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.b", 2024-09-17T04:33:54.444 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBRBulmsymlBBAAgBYGOXNn7aSEHMxkvsewJQ==", 2024-09-17T04:33:54.444 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-17T04:33:54.444 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-17T04:33:54.445 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-17T04:33:54.445 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-17T04:33:54.445 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-17T04:33:54.445 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:54.445 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:54.445 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T04:33:54.445 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.c", 2024-09-17T04:33:54.445 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBQBulmjaCsOhAA3c9mpeC0D8K9utnc6/13zw==", 2024-09-17T04:33:54.445 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-17T04:33:54.445 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-17T04:33:54.445 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-17T04:33:54.445 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-17T04:33:54.445 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-17T04:33:54.445 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:54.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:54.446 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T04:33:54.446 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.d", 2024-09-17T04:33:54.446 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBRBulmnIlcDBAAXbqz9EnlFmxAb2E8OO9/KQ==", 2024-09-17T04:33:54.446 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-17T04:33:54.446 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-17T04:33:54.446 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-17T04:33:54.446 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-17T04:33:54.446 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-17T04:33:54.446 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:54.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:54.446 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T04:33:54.446 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.0", 2024-09-17T04:33:54.446 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBUBulm51ntLBAA0Hm4yP85dCj3Ee10jnYVjw==", 2024-09-17T04:33:54.447 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-17T04:33:54.447 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-17T04:33:54.447 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-17T04:33:54.447 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-17T04:33:54.447 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:54.447 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:54.447 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T04:33:54.447 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.1", 2024-09-17T04:33:54.447 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBYBulmYkPXAhAA5+ybQVYsBLT2FkgWOzoxNA==", 2024-09-17T04:33:54.447 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-17T04:33:54.447 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-17T04:33:54.447 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-17T04:33:54.447 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-17T04:33:54.447 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:54.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:54.448 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T04:33:54.448 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.2", 2024-09-17T04:33:54.448 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBbBulmcpcoIxAAFy7BHRdyJ7X+bCpf/LYOLg==", 2024-09-17T04:33:54.448 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-17T04:33:54.448 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-17T04:33:54.448 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-17T04:33:54.448 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-17T04:33:54.448 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:54.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:54.448 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T04:33:54.448 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.3", 2024-09-17T04:33:54.449 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBeBulmYt3XJhAA9N1FTXul2rk6U1bkQ5VraQ==", 2024-09-17T04:33:54.449 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-17T04:33:54.449 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-17T04:33:54.449 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-17T04:33:54.449 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-17T04:33:54.449 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:54.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:54.449 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T04:33:54.449 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.4", 2024-09-17T04:33:54.449 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBlBulm4E2QBxAARmAT4UEHND01x5vb8fHb0w==", 2024-09-17T04:33:54.449 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-17T04:33:54.449 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-17T04:33:54.450 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-17T04:33:54.450 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-17T04:33:54.450 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:54.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:54.450 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T04:33:54.450 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.5", 2024-09-17T04:33:54.450 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBoBulm6BPtJxAArpGCRHadXl1WNGsyM19FUg==", 2024-09-17T04:33:54.450 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-17T04:33:54.450 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-17T04:33:54.450 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-17T04:33:54.450 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-17T04:33:54.450 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:54.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:54.451 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T04:33:54.451 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.6", 2024-09-17T04:33:54.451 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBsBulmBt8/DBAAk587ed/x3qfWy9b0iPYSqA==", 2024-09-17T04:33:54.451 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-17T04:33:54.451 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-17T04:33:54.451 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-17T04:33:54.451 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-17T04:33:54.452 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:54.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:54.452 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T04:33:54.452 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.7", 2024-09-17T04:33:54.452 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBvBulmLSjWHRAA7crk7TbyBJ1ihfqGMd/G1w==", 2024-09-17T04:33:54.452 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-17T04:33:54.453 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-09-17T04:33:54.453 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-09-17T04:33:54.453 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-17T04:33:54.453 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:54.453 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:54.453 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T04:33:54.454 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.0", 2024-09-17T04:33:54.454 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBRBulm3vQYERAA/gAxWQECQvkbphM8KrWgUQ==", 2024-09-17T04:33:54.454 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-17T04:33:54.454 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-17T04:33:54.454 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-09-17T04:33:54.454 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-09-17T04:33:54.454 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-09-17T04:33:54.455 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:54.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:54.455 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T04:33:54.455 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.1", 2024-09-17T04:33:54.455 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBRBulmh13QGBAAhN/ly2QhbEKp3oAclmXC4w==", 2024-09-17T04:33:54.455 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-17T04:33:54.456 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-17T04:33:54.456 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-09-17T04:33:54.456 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-09-17T04:33:54.456 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-09-17T04:33:54.456 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:54.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:54.457 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T04:33:54.457 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.2", 2024-09-17T04:33:54.457 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBRBulmQnAHFRAAujq2Mnektuzbv+nd8v+CTg==", 2024-09-17T04:33:54.457 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-17T04:33:54.457 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-17T04:33:54.457 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-09-17T04:33:54.458 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-09-17T04:33:54.458 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-09-17T04:33:54.458 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:54.458 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:54.458 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T04:33:54.459 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.3", 2024-09-17T04:33:54.459 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBRBulmdqWRHBAAE91kKLf4sVqvfbS2oxFHfg==", 2024-09-17T04:33:54.459 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-17T04:33:54.459 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-09-17T04:33:54.459 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-09-17T04:33:54.459 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-09-17T04:33:54.460 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-09-17T04:33:54.460 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:54.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:54.460 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T04:33:54.460 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.admin", 2024-09-17T04:33:54.461 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBQBulm/OM6FBAAgU6AaDfEjmjqmgnoJnCncA==", 2024-09-17T04:33:54.461 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-17T04:33:54.461 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-09-17T04:33:54.461 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-09-17T04:33:54.461 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-09-17T04:33:54.461 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-17T04:33:54.462 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:54.462 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:54.462 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T04:33:54.462 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-mds", 2024-09-17T04:33:54.462 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQB2BulmcyFZCBAA84HAqJ3die/qYroY0q8/2Q==", 2024-09-17T04:33:54.462 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-17T04:33:54.463 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-mds" 2024-09-17T04:33:54.463 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:54.463 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:54.463 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T04:33:54.463 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-mgr", 2024-09-17T04:33:54.463 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQB2BulmQipZCBAAH9ImIgBaC0ffdVVcX9EZoA==", 2024-09-17T04:33:54.464 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-17T04:33:54.464 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-17T04:33:54.464 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:54.464 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:54.464 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T04:33:54.465 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-osd", 2024-09-17T04:33:54.465 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQB2BulmQTJZCBAA3KpAPL8qMV5u+vDav63OxQ==", 2024-09-17T04:33:54.465 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-17T04:33:54.466 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-osd" 2024-09-17T04:33:54.466 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:54.467 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:54.467 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T04:33:54.467 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-rbd", 2024-09-17T04:33:54.467 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQB2Bulm1zlZCBAAeZYqBn9CUc2Ji5MU2KBddA==", 2024-09-17T04:33:54.467 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-17T04:33:54.468 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-17T04:33:54.468 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:54.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:54.468 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T04:33:54.468 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-17T04:33:54.468 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQB2BulmfkFZCBAAgg1qDu+7jMFm3MLNlvMDxQ==", 2024-09-17T04:33:54.469 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-17T04:33:54.469 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-17T04:33:54.469 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:54.469 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:54.469 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T04:33:54.469 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-rgw", 2024-09-17T04:33:54.470 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQB2BulmY0lZCBAAwe+HTCLLlNr9gdoP6QwUMg==", 2024-09-17T04:33:54.470 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-17T04:33:54.470 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-17T04:33:54.470 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:54.470 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:54.470 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T04:33:54.471 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.x", 2024-09-17T04:33:54.471 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBQBulmL235LhAAMvjhUWzQszGESkHd4NlKYQ==", 2024-09-17T04:33:54.471 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-17T04:33:54.471 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-09-17T04:33:54.471 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-09-17T04:33:54.471 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-17T04:33:54.472 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:54.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:33:54.472 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T04:33:54.472 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.y", 2024-09-17T04:33:54.472 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBQBulmow4mKhAAhECLmAc2tfC/HhfQsOGxeQ==", 2024-09-17T04:33:54.473 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-09-17T04:33:54.473 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-09-17T04:33:54.473 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-09-17T04:33:54.473 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-09-17T04:33:54.473 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:54.473 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:54.474 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T04:33:54.474 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T04:33:54.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.445+0000 7f9cf83c3640 1 -- 172.21.15.37:0/3288018852 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9cd4078bc0 msgr2=0x7f9cd407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:54.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.445+0000 7f9cf83c3640 1 --2- 172.21.15.37:0/3288018852 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9cd4078bc0 0x7f9cd407b0d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9cec00a490 tx=0x7f9cec00a420 comp rx=0 tx=0).stop 2024-09-17T04:33:54.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.445+0000 7f9cf83c3640 1 -- 172.21.15.37:0/3288018852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf015c730 msgr2=0x7f9cf0160a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:54.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.445+0000 7f9cf83c3640 1 --2- 172.21.15.37:0/3288018852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf015c730 0x7f9cf0160a20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9ce800c6c0 tx=0x7f9ce800cb80 comp rx=0 tx=0).stop 2024-09-17T04:33:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.446+0000 7f9cf83c3640 1 -- 172.21.15.37:0/3288018852 shutdown_connections 2024-09-17T04:33:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.446+0000 7f9cf83c3640 1 --2- 172.21.15.37:0/3288018852 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9cd4078bc0 0x7f9cd407b0d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.446+0000 7f9cf83c3640 1 --2- 172.21.15.37:0/3288018852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf015c730 0x7f9cf0160a20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.446+0000 7f9cf83c3640 1 --2- 172.21.15.37:0/3288018852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cf00b2390 0x7f9cf0158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.446+0000 7f9cf83c3640 1 --2- 172.21.15.37:0/3288018852 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9cf00b1f80 0x7f9cf01f4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:54.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.446+0000 7f9cf83c3640 1 -- 172.21.15.37:0/3288018852 >> 172.21.15.37:0/3288018852 conn(0x7f9cf00d1300 msgr2=0x7f9cf00b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:54.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.446+0000 7f9cf83c3640 1 -- 172.21.15.37:0/3288018852 shutdown_connections 2024-09-17T04:33:54.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.446+0000 7f9cf83c3640 1 -- 172.21.15.37:0/3288018852 wait complete. 2024-09-17T04:33:54.480 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-17T04:33:54.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.600+0000 7fc658e8b640 1 -- 172.21.15.37:0/3002261006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc63c004800 msgr2=0x7fc63c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:54.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.600+0000 7fc658e8b640 1 --2- 172.21.15.37:0/3002261006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc63c004800 0x7fc63c004cd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc648006f40 tx=0x7fc63c005830 comp rx=0 tx=0).stop 2024-09-17T04:33:54.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.600+0000 7fc652d9d640 1 -- 172.21.15.37:0/3002261006 reap_dead start 2024-09-17T04:33:54.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.600+0000 7fc658e8b640 1 -- 172.21.15.37:0/3002261006 shutdown_connections 2024-09-17T04:33:54.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.600+0000 7fc658e8b640 1 -- 172.21.15.37:0/3002261006 >> 172.21.15.37:0/3002261006 conn(0x7fc654072a80 msgr2=0x7fc654072e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:54.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.600+0000 7fc658e8b640 1 -- 172.21.15.37:0/3002261006 shutdown_connections 2024-09-17T04:33:54.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.600+0000 7fc658e8b640 1 -- 172.21.15.37:0/3002261006 wait complete. 2024-09-17T04:33:54.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.602+0000 7fc658e8b640 1 Processor -- start 2024-09-17T04:33:54.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.602+0000 7fc658e8b640 1 -- start start 2024-09-17T04:33:54.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.603+0000 7fc658e8b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc63c004800 0x7fc6541431c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:54.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.603+0000 7fc658e8b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc6541f62f0 0x7fc6541f6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:54.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.603+0000 7fc658e8b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6541f6c40 0x7fc654143df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:54.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.603+0000 7fc658e8b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc6540b7d50 con 0x7fc6541f6c40 2024-09-17T04:33:54.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.603+0000 7fc658e8b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc6540cd4e0 con 0x7fc63c004800 2024-09-17T04:33:54.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.603+0000 7fc658e8b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fc6540b7ed0 con 0x7fc6541f62f0 2024-09-17T04:33:54.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.603+0000 7fc652d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6541f6c40 0x7fc654143df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:54.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.603+0000 7fc652d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6541f6c40 0x7fc654143df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49508/0 (socket says 172.21.15.37:49508) 2024-09-17T04:33:54.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.603+0000 7fc652d9d640 1 -- 172.21.15.37:0/3666553472 learned_addr learned my addr 172.21.15.37:0/3666553472 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.604+0000 7fc65259c640 1 --2- 172.21.15.37:0/3666553472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc63c004800 0x7fc6541431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.604+0000 7fc651d9b640 1 --2- 172.21.15.37:0/3666553472 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc6541f62f0 0x7fc6541f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.604+0000 7fc652d9d640 1 -- 172.21.15.37:0/3666553472 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc6541f62f0 msgr2=0x7fc6541f6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.604+0000 7fc652d9d640 1 --2- 172.21.15.37:0/3666553472 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc6541f62f0 0x7fc6541f6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.604+0000 7fc652d9d640 1 -- 172.21.15.37:0/3666553472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc63c004800 msgr2=0x7fc6541431c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.604+0000 7fc652d9d640 1 --2- 172.21.15.37:0/3666553472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc63c004800 0x7fc6541431c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:54.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.604+0000 7fc652d9d640 1 -- 172.21.15.37:0/3666553472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc654155770 con 0x7fc6541f6c40 2024-09-17T04:33:54.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.604+0000 7fc65259c640 1 --2- 172.21.15.37:0/3666553472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc63c004800 0x7fc6541431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:54.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.604+0000 7fc651d9b640 1 --2- 172.21.15.37:0/3666553472 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc6541f62f0 0x7fc6541f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:33:54.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.604+0000 7fc652d9d640 1 --2- 172.21.15.37:0/3666553472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6541f6c40 0x7fc654143df0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc6540cd9e0 tx=0x7fc64c001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:54.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.604+0000 7fc642ffd640 1 -- 172.21.15.37:0/3666553472 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c011840 con 0x7fc6541f6c40 2024-09-17T04:33:54.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.604+0000 7fc642ffd640 1 -- 172.21.15.37:0/3666553472 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc64c018070 con 0x7fc6541f6c40 2024-09-17T04:33:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.604+0000 7fc658e8b640 1 -- 172.21.15.37:0/3666553472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc654155a50 con 0x7fc6541f6c40 2024-09-17T04:33:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.604+0000 7fc642ffd640 1 -- 172.21.15.37:0/3666553472 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c011c60 con 0x7fc6541f6c40 2024-09-17T04:33:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.604+0000 7fc658e8b640 1 -- 172.21.15.37:0/3666553472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc654156010 con 0x7fc6541f6c40 2024-09-17T04:33:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.605+0000 7fc658e8b640 1 -- 172.21.15.37:0/3666553472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6540760f0 con 0x7fc6541f6c40 2024-09-17T04:33:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.606+0000 7fc642ffd640 1 -- 172.21.15.37:0/3666553472 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc64c01d460 con 0x7fc6541f6c40 2024-09-17T04:33:54.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.607+0000 7fc642ffd640 1 --2- 172.21.15.37:0/3666553472 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc634078b90 0x7fc63407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:54.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.607+0000 7fc642ffd640 1 -- 172.21.15.37:0/3666553472 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fc64c09aff0 con 0x7fc6541f6c40 2024-09-17T04:33:54.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.607+0000 7fc65259c640 1 --2- 172.21.15.37:0/3666553472 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc634078b90 0x7fc63407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:54.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.608+0000 7fc65259c640 1 --2- 172.21.15.37:0/3666553472 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc634078b90 0x7fc63407b0a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc644005fd0 tx=0x7fc644009500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:54.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.611+0000 7fc642ffd640 1 -- 172.21.15.37:0/3666553472 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc64c063810 con 0x7fc6541f6c40 2024-09-17T04:33:54.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.871+0000 7fc658e8b640 1 -- 172.21.15.37:0/3666553472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fc654155500 con 0x7fc6541f6c40 2024-09-17T04:33:54.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.873+0000 7fc642ffd640 1 -- 172.21.15.37:0/3666553472 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v11) ==== 90+0+0 (secure 0 0 0) 0x7fc64c063110 con 0x7fc6541f6c40 2024-09-17T04:33:54.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.876+0000 7fc658e8b640 1 -- 172.21.15.37:0/3666553472 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc634078b90 msgr2=0x7fc63407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:54.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.876+0000 7fc658e8b640 1 --2- 172.21.15.37:0/3666553472 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc634078b90 0x7fc63407b0a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc644005fd0 tx=0x7fc644009500 comp rx=0 tx=0).stop 2024-09-17T04:33:54.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.876+0000 7fc658e8b640 1 -- 172.21.15.37:0/3666553472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6541f6c40 msgr2=0x7fc654143df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:54.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.876+0000 7fc658e8b640 1 --2- 172.21.15.37:0/3666553472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6541f6c40 0x7fc654143df0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc6540cd9e0 tx=0x7fc64c001800 comp rx=0 tx=0).stop 2024-09-17T04:33:54.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.876+0000 7fc658e8b640 1 -- 172.21.15.37:0/3666553472 shutdown_connections 2024-09-17T04:33:54.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.876+0000 7fc658e8b640 1 --2- 172.21.15.37:0/3666553472 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc634078b90 0x7fc63407b0a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:54.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.876+0000 7fc658e8b640 1 --2- 172.21.15.37:0/3666553472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6541f6c40 0x7fc654143df0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:54.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.876+0000 7fc658e8b640 1 --2- 172.21.15.37:0/3666553472 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc6541f62f0 0x7fc6541f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:54.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.876+0000 7fc658e8b640 1 --2- 172.21.15.37:0/3666553472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc63c004800 0x7fc6541431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:54.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.876+0000 7fc658e8b640 1 -- 172.21.15.37:0/3666553472 >> 172.21.15.37:0/3666553472 conn(0x7fc654072a80 msgr2=0x7fc65415dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:54.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.876+0000 7fc658e8b640 1 -- 172.21.15.37:0/3666553472 shutdown_connections 2024-09-17T04:33:54.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:54.876+0000 7fc658e8b640 1 -- 172.21.15.37:0/3666553472 wait complete. 2024-09-17T04:33:54.888 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-17T04:33:55.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.006+0000 7f61b68de640 1 -- 172.21.15.37:0/770921401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61980051a0 msgr2=0x7f619800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:55.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.006+0000 7f61b68de640 1 --2- 172.21.15.37:0/770921401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61980051a0 0x7f619800b9c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f61a8001e40 tx=0x7f6198005830 comp rx=0 tx=0).stop 2024-09-17T04:33:55.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.006+0000 7f61b4e7b640 1 -- 172.21.15.37:0/770921401 reap_dead start 2024-09-17T04:33:55.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.006+0000 7f61b68de640 1 -- 172.21.15.37:0/770921401 shutdown_connections 2024-09-17T04:33:55.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.006+0000 7f61b68de640 1 -- 172.21.15.37:0/770921401 >> 172.21.15.37:0/770921401 conn(0x7f61b00d44c0 msgr2=0x7f61b00d48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:55.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.006+0000 7f61b68de640 1 -- 172.21.15.37:0/770921401 shutdown_connections 2024-09-17T04:33:55.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.006+0000 7f61b68de640 1 -- 172.21.15.37:0/770921401 wait complete. 2024-09-17T04:33:55.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.007+0000 7f61b68de640 1 Processor -- start 2024-09-17T04:33:55.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.008+0000 7f61b68de640 1 -- start start 2024-09-17T04:33:55.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.009+0000 7f61b68de640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61b015f500 0x7f61b01f8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:55.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.009+0000 7f61b68de640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61b0153a50 0x7f61b01f5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:55.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.009+0000 7f61b68de640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f61b00cc740 0x7f61b00ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:55.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.009+0000 7f61b68de640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f61b015e8e0 con 0x7f61b015f500 2024-09-17T04:33:55.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.009+0000 7f61b68de640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f61b00d41f0 con 0x7f61b0153a50 2024-09-17T04:33:55.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.009+0000 7f61b68de640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f61b015ea60 con 0x7f61b00cc740 2024-09-17T04:33:55.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.009+0000 7f61affff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61b015f500 0x7f61b01f8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:55.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.009+0000 7f61affff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61b015f500 0x7f61b01f8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49524/0 (socket says 172.21.15.37:49524) 2024-09-17T04:33:55.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.009+0000 7f61affff640 1 -- 172.21.15.37:0/120300918 learned_addr learned my addr 172.21.15.37:0/120300918 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:55.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.009+0000 7f61af7fe640 1 --2- 172.21.15.37:0/120300918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61b0153a50 0x7f61b01f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:55.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.009+0000 7f61b4e7b640 1 --2- 172.21.15.37:0/120300918 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f61b00cc740 0x7f61b00ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:55.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.009+0000 7f61affff640 1 -- 172.21.15.37:0/120300918 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f61b00cc740 msgr2=0x7f61b00ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:55.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.009+0000 7f61affff640 1 --2- 172.21.15.37:0/120300918 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f61b00cc740 0x7f61b00ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.009+0000 7f61affff640 1 -- 172.21.15.37:0/120300918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61b0153a50 msgr2=0x7f61b01f5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:55.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.009+0000 7f61affff640 1 --2- 172.21.15.37:0/120300918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61b0153a50 0x7f61b01f5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.010+0000 7f61affff640 1 -- 172.21.15.37:0/120300918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b015f910 con 0x7f61b015f500 2024-09-17T04:33:55.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.010+0000 7f61affff640 1 --2- 172.21.15.37:0/120300918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61b015f500 0x7f61b01f8e00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f61a0011a50 tx=0x7f61a0011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:55.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.010+0000 7f61acff9640 1 -- 172.21.15.37:0/120300918 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a001f070 con 0x7f61b015f500 2024-09-17T04:33:55.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.010+0000 7f61b68de640 1 -- 172.21.15.37:0/120300918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b015fb90 con 0x7f61b015f500 2024-09-17T04:33:55.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.010+0000 7f61acff9640 1 -- 172.21.15.37:0/120300918 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61a000fdc0 con 0x7f61b015f500 2024-09-17T04:33:55.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.010+0000 7f61acff9640 1 -- 172.21.15.37:0/120300918 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a0012690 con 0x7f61b015f500 2024-09-17T04:33:55.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.010+0000 7f61b68de640 1 -- 172.21.15.37:0/120300918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b0160150 con 0x7f61b015f500 2024-09-17T04:33:55.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.011+0000 7f61b68de640 1 -- 172.21.15.37:0/120300918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61b0160390 con 0x7f61b015f500 2024-09-17T04:33:55.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.012+0000 7f61acff9640 1 -- 172.21.15.37:0/120300918 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f61a0022070 con 0x7f61b015f500 2024-09-17T04:33:55.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.012+0000 7f61acff9640 1 --2- 172.21.15.37:0/120300918 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f619c078bc0 0x7f619c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:55.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.012+0000 7f61acff9640 1 -- 172.21.15.37:0/120300918 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f61a009b390 con 0x7f61b015f500 2024-09-17T04:33:55.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.013+0000 7f61af7fe640 1 --2- 172.21.15.37:0/120300918 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f619c078bc0 0x7f619c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:55.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.013+0000 7f61af7fe640 1 --2- 172.21.15.37:0/120300918 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f619c078bc0 0x7f619c07b0d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f61b00b7f80 tx=0x7f61a4001e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:55.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.017+0000 7f61acff9640 1 -- 172.21.15.37:0/120300918 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f61b0160390 con 0x7f61b015f500 2024-09-17T04:33:55.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.279+0000 7f61b68de640 1 -- 172.21.15.37:0/120300918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f61b00b5f40 con 0x7f61b015f500 2024-09-17T04:33:55.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.282+0000 7f61acff9640 1 -- 172.21.15.37:0/120300918 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7f61b015f910 con 0x7f61b015f500 2024-09-17T04:33:55.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.284+0000 7f61b68de640 1 -- 172.21.15.37:0/120300918 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f619c078bc0 msgr2=0x7f619c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:55.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.284+0000 7f61b68de640 1 --2- 172.21.15.37:0/120300918 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f619c078bc0 0x7f619c07b0d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f61b00b7f80 tx=0x7f61a4001e70 comp rx=0 tx=0).stop 2024-09-17T04:33:55.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.284+0000 7f61b68de640 1 -- 172.21.15.37:0/120300918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61b015f500 msgr2=0x7f61b01f8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:55.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.284+0000 7f61b68de640 1 --2- 172.21.15.37:0/120300918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61b015f500 0x7f61b01f8e00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f61a0011a50 tx=0x7f61a0011e50 comp rx=0 tx=0).stop 2024-09-17T04:33:55.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.284+0000 7f61b68de640 1 -- 172.21.15.37:0/120300918 shutdown_connections 2024-09-17T04:33:55.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.284+0000 7f61b68de640 1 --2- 172.21.15.37:0/120300918 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f619c078bc0 0x7f619c07b0d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.284+0000 7f61b68de640 1 --2- 172.21.15.37:0/120300918 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f61b00cc740 0x7f61b00ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.284+0000 7f61b68de640 1 --2- 172.21.15.37:0/120300918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61b0153a50 0x7f61b01f5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.284+0000 7f61b68de640 1 --2- 172.21.15.37:0/120300918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61b015f500 0x7f61b01f8e00 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.284+0000 7f61b68de640 1 -- 172.21.15.37:0/120300918 >> 172.21.15.37:0/120300918 conn(0x7f61b00d44c0 msgr2=0x7f61b00b7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:55.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.284+0000 7f61b68de640 1 -- 172.21.15.37:0/120300918 shutdown_connections 2024-09-17T04:33:55.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.284+0000 7f61b68de640 1 -- 172.21.15.37:0/120300918 wait complete. 2024-09-17T04:33:55.295 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-17T04:33:55.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.412+0000 7f73506e5640 1 -- 172.21.15.37:0/857671090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73480d1a10 msgr2=0x7f73480ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:55.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.412+0000 7f73506e5640 1 --2- 172.21.15.37:0/857671090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73480d1a10 0x7f73480ccb50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f733c005fd0 tx=0x7f733c02a5d0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.413+0000 7f73506e5640 1 -- 172.21.15.37:0/857671090 shutdown_connections 2024-09-17T04:33:55.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.413+0000 7f73506e5640 1 --2- 172.21.15.37:0/857671090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73480d1a10 0x7f73480ccb50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.413+0000 7f73506e5640 1 --2- 172.21.15.37:0/857671090 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f734815c830 0x7f73480d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.413+0000 7f73506e5640 1 -- 172.21.15.37:0/857671090 >> 172.21.15.37:0/857671090 conn(0x7f7348071050 msgr2=0x7f7348071460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:55.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.413+0000 7f73506e5640 1 -- 172.21.15.37:0/857671090 shutdown_connections 2024-09-17T04:33:55.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.413+0000 7f73506e5640 1 -- 172.21.15.37:0/857671090 wait complete. 2024-09-17T04:33:55.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.414+0000 7f73506e5640 1 Processor -- start 2024-09-17T04:33:55.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.414+0000 7f73506e5640 1 -- start start 2024-09-17T04:33:55.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.416+0000 7f73506e5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73480cd090 0x7f73481f30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:55.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.416+0000 7f73506e5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73480d1a10 0x7f7348154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:55.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.416+0000 7f73506e5640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f734815c830 0x7f73481591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:55.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.416+0000 7f73506e5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7348073080 con 0x7f73480d1a10 2024-09-17T04:33:55.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.416+0000 7f73506e5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7348072f00 con 0x7f73480cd090 2024-09-17T04:33:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.416+0000 7f73506e5640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f7348073200 con 0x7f734815c830 2024-09-17T04:33:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.416+0000 7f734dc80640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73480d1a10 0x7f7348154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.416+0000 7f734dc80640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73480d1a10 0x7f7348154a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34766/0 (socket says 172.21.15.37:34766) 2024-09-17T04:33:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.416+0000 7f734dc80640 1 -- 172.21.15.37:0/3738258187 learned_addr learned my addr 172.21.15.37:0/3738258187 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.416+0000 7f734e481640 1 --2- 172.21.15.37:0/3738258187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73480cd090 0x7f73481f30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.416+0000 7f734ec82640 1 --2- 172.21.15.37:0/3738258187 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f734815c830 0x7f73481591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.416+0000 7f734dc80640 1 -- 172.21.15.37:0/3738258187 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f734815c830 msgr2=0x7f73481591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.416+0000 7f734dc80640 1 --2- 172.21.15.37:0/3738258187 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f734815c830 0x7f73481591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.416+0000 7f734dc80640 1 -- 172.21.15.37:0/3738258187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73480cd090 msgr2=0x7f73481f30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.416+0000 7f734dc80640 1 --2- 172.21.15.37:0/3738258187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73480cd090 0x7f73481f30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.416+0000 7f734dc80640 1 -- 172.21.15.37:0/3738258187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73481559e0 con 0x7f73480d1a10 2024-09-17T04:33:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.416+0000 7f734dc80640 1 --2- 172.21.15.37:0/3738258187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73480d1a10 0x7f7348154a60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f733c034040 tx=0x7f733c0054c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.416+0000 7f733affd640 1 -- 172.21.15.37:0/3738258187 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c03e030 con 0x7f73480d1a10 2024-09-17T04:33:55.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.417+0000 7f733affd640 1 -- 172.21.15.37:0/3738258187 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f733c031610 con 0x7f73480d1a10 2024-09-17T04:33:55.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.417+0000 7f73506e5640 1 -- 172.21.15.37:0/3738258187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7348155cc0 con 0x7f73480d1a10 2024-09-17T04:33:55.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.417+0000 7f733affd640 1 -- 172.21.15.37:0/3738258187 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c035040 con 0x7f73480d1a10 2024-09-17T04:33:55.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.417+0000 7f73506e5640 1 -- 172.21.15.37:0/3738258187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7348156280 con 0x7f73480d1a10 2024-09-17T04:33:55.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.418+0000 7f73506e5640 1 -- 172.21.15.37:0/3738258187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73480ccb50 con 0x7f73480d1a10 2024-09-17T04:33:55.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.418+0000 7f733affd640 1 -- 172.21.15.37:0/3738258187 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f733c041030 con 0x7f73480d1a10 2024-09-17T04:33:55.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.419+0000 7f733affd640 1 --2- 172.21.15.37:0/3738258187 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f732c078b90 0x7f732c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:55.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.419+0000 7f733affd640 1 -- 172.21.15.37:0/3738258187 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f733c0bafb0 con 0x7f73480d1a10 2024-09-17T04:33:55.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.419+0000 7f734e481640 1 --2- 172.21.15.37:0/3738258187 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f732c078b90 0x7f732c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:55.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.420+0000 7f734e481640 1 --2- 172.21.15.37:0/3738258187 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f732c078b90 0x7f732c07b0a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7344005fd0 tx=0x7f734400a250 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:55.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.424+0000 7f733affd640 1 -- 172.21.15.37:0/3738258187 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f733c083180 con 0x7f73480d1a10 2024-09-17T04:33:55.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.682+0000 7f73506e5640 1 -- 172.21.15.37:0/3738258187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f73480d20b0 con 0x7f73480d1a10 2024-09-17T04:33:55.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.684+0000 7f733affd640 1 -- 172.21.15.37:0/3738258187 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v13) ==== 90+0+0 (secure 0 0 0) 0x7f733c087bb0 con 0x7f73480d1a10 2024-09-17T04:33:55.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.686+0000 7f73506e5640 1 -- 172.21.15.37:0/3738258187 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f732c078b90 msgr2=0x7f732c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.686+0000 7f73506e5640 1 --2- 172.21.15.37:0/3738258187 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f732c078b90 0x7f732c07b0a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7344005fd0 tx=0x7f734400a250 comp rx=0 tx=0).stop 2024-09-17T04:33:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.686+0000 7f73506e5640 1 -- 172.21.15.37:0/3738258187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73480d1a10 msgr2=0x7f7348154a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.686+0000 7f73506e5640 1 --2- 172.21.15.37:0/3738258187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73480d1a10 0x7f7348154a60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f733c034040 tx=0x7f733c0054c0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.686+0000 7f73506e5640 1 -- 172.21.15.37:0/3738258187 shutdown_connections 2024-09-17T04:33:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.686+0000 7f73506e5640 1 --2- 172.21.15.37:0/3738258187 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f732c078b90 0x7f732c07b0a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.686+0000 7f73506e5640 1 --2- 172.21.15.37:0/3738258187 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f734815c830 0x7f73481591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.686+0000 7f73506e5640 1 --2- 172.21.15.37:0/3738258187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73480d1a10 0x7f7348154a60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.686+0000 7f73506e5640 1 --2- 172.21.15.37:0/3738258187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73480cd090 0x7f73481f30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.686+0000 7f73506e5640 1 -- 172.21.15.37:0/3738258187 >> 172.21.15.37:0/3738258187 conn(0x7f7348071050 msgr2=0x7f73480d28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.687+0000 7f73506e5640 1 -- 172.21.15.37:0/3738258187 shutdown_connections 2024-09-17T04:33:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.687+0000 7f73506e5640 1 -- 172.21.15.37:0/3738258187 wait complete. 2024-09-17T04:33:55.698 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-17T04:33:55.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.815+0000 7f23b5abd640 1 -- 172.21.15.37:0/2091143656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23b0162cf0 msgr2=0x7f23b00d2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:55.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.815+0000 7f23b5abd640 1 --2- 172.21.15.37:0/2091143656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23b0162cf0 0x7f23b00d2650 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f23a8005fd0 tx=0x7f23a802aa00 comp rx=0 tx=0).stop 2024-09-17T04:33:55.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.815+0000 7f23b5abd640 1 -- 172.21.15.37:0/2091143656 shutdown_connections 2024-09-17T04:33:55.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.815+0000 7f23b5abd640 1 --2- 172.21.15.37:0/2091143656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23b0162cf0 0x7f23b00d2650 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.815+0000 7f23b5abd640 1 --2- 172.21.15.37:0/2091143656 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f23b00b66d0 0x7f23b01627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.815+0000 7f23b5abd640 1 -- 172.21.15.37:0/2091143656 >> 172.21.15.37:0/2091143656 conn(0x7f23b00d4540 msgr2=0x7f23b00d4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:55.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.815+0000 7f23b5abd640 1 -- 172.21.15.37:0/2091143656 shutdown_connections 2024-09-17T04:33:55.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.815+0000 7f23b5abd640 1 -- 172.21.15.37:0/2091143656 wait complete. 2024-09-17T04:33:55.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.817+0000 7f23b5abd640 1 Processor -- start 2024-09-17T04:33:55.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.817+0000 7f23b5abd640 1 -- start start 2024-09-17T04:33:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.818+0000 7f23b5abd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23b015e830 0x7f23b00b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.819+0000 7f23b5abd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f23b0162cf0 0x7f23b015b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.819+0000 7f23b5abd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23b00b9070 0x7f23b00b9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.819+0000 7f23b5abd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f23b00d3090 con 0x7f23b015e830 2024-09-17T04:33:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.819+0000 7f23b5abd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f23b00d2f10 con 0x7f23b00b9070 2024-09-17T04:33:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.819+0000 7f23b5abd640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f23b00d3210 con 0x7f23b0162cf0 2024-09-17T04:33:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.819+0000 7f23af7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23b015e830 0x7f23b00b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.819+0000 7f23af7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23b015e830 0x7f23b00b2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34780/0 (socket says 172.21.15.37:34780) 2024-09-17T04:33:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.819+0000 7f23af7fe640 1 -- 172.21.15.37:0/3610271700 learned_addr learned my addr 172.21.15.37:0/3610271700 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.819+0000 7f23affff640 1 --2- 172.21.15.37:0/3610271700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23b00b9070 0x7f23b00b9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.819+0000 7f23aeffd640 1 --2- 172.21.15.37:0/3610271700 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f23b0162cf0 0x7f23b015b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.819+0000 7f23af7fe640 1 -- 172.21.15.37:0/3610271700 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f23b0162cf0 msgr2=0x7f23b015b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.819+0000 7f23af7fe640 1 --2- 172.21.15.37:0/3610271700 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f23b0162cf0 0x7f23b015b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.819+0000 7f23af7fe640 1 -- 172.21.15.37:0/3610271700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23b00b9070 msgr2=0x7f23b00b9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.819+0000 7f23af7fe640 1 --2- 172.21.15.37:0/3610271700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23b00b9070 0x7f23b00b9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.819+0000 7f23af7fe640 1 -- 172.21.15.37:0/3610271700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b00b99c0 con 0x7f23b015e830 2024-09-17T04:33:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.819+0000 7f23af7fe640 1 --2- 172.21.15.37:0/3610271700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23b015e830 0x7f23b00b2a50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f23a000b3f0 tx=0x7f23a000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.819+0000 7f2397fff640 1 -- 172.21.15.37:0/3610271700 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a0015020 con 0x7f23b015e830 2024-09-17T04:33:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.819+0000 7f2397fff640 1 -- 172.21.15.37:0/3610271700 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23a0011920 con 0x7f23b015e830 2024-09-17T04:33:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.819+0000 7f23b5abd640 1 -- 172.21.15.37:0/3610271700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23b00ce370 con 0x7f23b015e830 2024-09-17T04:33:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.820+0000 7f2397fff640 1 -- 172.21.15.37:0/3610271700 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a0011d70 con 0x7f23b015e830 2024-09-17T04:33:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.820+0000 7f23b5abd640 1 -- 172.21.15.37:0/3610271700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23b00cb0f0 con 0x7f23b015e830 2024-09-17T04:33:55.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.820+0000 7f23b5abd640 1 -- 172.21.15.37:0/3610271700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23b00cb330 con 0x7f23b015e830 2024-09-17T04:33:55.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.821+0000 7f2397fff640 1 -- 172.21.15.37:0/3610271700 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f23a0018020 con 0x7f23b015e830 2024-09-17T04:33:55.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.822+0000 7f2397fff640 1 --2- 172.21.15.37:0/3610271700 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f239c078bc0 0x7f239c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:55.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.822+0000 7f2397fff640 1 -- 172.21.15.37:0/3610271700 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f23a009a830 con 0x7f23b015e830 2024-09-17T04:33:55.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.822+0000 7f23aeffd640 1 --2- 172.21.15.37:0/3610271700 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f239c078bc0 0x7f239c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:55.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.823+0000 7f23aeffd640 1 --2- 172.21.15.37:0/3610271700 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f239c078bc0 0x7f239c07b0d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f23b00b22c0 tx=0x7f23a4005f60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:55.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:55.826+0000 7f2397fff640 1 -- 172.21.15.37:0/3610271700 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f23b00cb330 con 0x7f23b015e830 2024-09-17T04:33:56.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.085+0000 7f23b5abd640 1 -- 172.21.15.37:0/3610271700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f23b00d36e0 con 0x7f23b015e830 2024-09-17T04:33:56.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.087+0000 7f2397fff640 1 -- 172.21.15.37:0/3610271700 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v14) ==== 90+0+0 (secure 0 0 0) 0x7f23a001f020 con 0x7f23b015e830 2024-09-17T04:33:56.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.090+0000 7f23b5abd640 1 -- 172.21.15.37:0/3610271700 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f239c078bc0 msgr2=0x7f239c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:56.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.090+0000 7f23b5abd640 1 --2- 172.21.15.37:0/3610271700 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f239c078bc0 0x7f239c07b0d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f23b00b22c0 tx=0x7f23a4005f60 comp rx=0 tx=0).stop 2024-09-17T04:33:56.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.090+0000 7f23b5abd640 1 -- 172.21.15.37:0/3610271700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23b015e830 msgr2=0x7f23b00b2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:56.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.090+0000 7f23b5abd640 1 --2- 172.21.15.37:0/3610271700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23b015e830 0x7f23b00b2a50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f23a000b3f0 tx=0x7f23a000b8b0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.090+0000 7f23b5abd640 1 -- 172.21.15.37:0/3610271700 shutdown_connections 2024-09-17T04:33:56.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.090+0000 7f23b5abd640 1 --2- 172.21.15.37:0/3610271700 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f239c078bc0 0x7f239c07b0d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.090+0000 7f23b5abd640 1 --2- 172.21.15.37:0/3610271700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23b00b9070 0x7f23b00b9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.090+0000 7f23b5abd640 1 --2- 172.21.15.37:0/3610271700 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f23b0162cf0 0x7f23b015b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.090+0000 7f23b5abd640 1 --2- 172.21.15.37:0/3610271700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23b015e830 0x7f23b00b2a50 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.090+0000 7f23b5abd640 1 -- 172.21.15.37:0/3610271700 >> 172.21.15.37:0/3610271700 conn(0x7f23b00d4540 msgr2=0x7f23b00b9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:56.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.090+0000 7f23b5abd640 1 -- 172.21.15.37:0/3610271700 shutdown_connections 2024-09-17T04:33:56.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.090+0000 7f23b5abd640 1 -- 172.21.15.37:0/3610271700 wait complete. 2024-09-17T04:33:56.101 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-17T04:33:56.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.220+0000 7fd3a6d50640 1 -- 172.21.15.37:0/39052118 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd3a00b6650 msgr2=0x7fd3a015e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:56.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.220+0000 7fd3a6d50640 1 --2- 172.21.15.37:0/39052118 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd3a00b6650 0x7fd3a015e8e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd390009830 tx=0x7fd39002a510 comp rx=0 tx=0).stop 2024-09-17T04:33:56.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.220+0000 7fd3a6d50640 1 -- 172.21.15.37:0/39052118 shutdown_connections 2024-09-17T04:33:56.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.220+0000 7fd3a6d50640 1 --2- 172.21.15.37:0/39052118 >> v2:172.21.15.92:3300/0 conn(0x7fd3a00cef40 0x7fd3a00b7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.220+0000 7fd3a6d50640 1 --2- 172.21.15.37:0/39052118 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd3a00b6650 0x7fd3a015e8e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.220+0000 7fd3a6d50640 1 -- 172.21.15.37:0/39052118 >> 172.21.15.37:0/39052118 conn(0x7fd3a00d44c0 msgr2=0x7fd3a00d48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:56.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.220+0000 7fd3a6d50640 1 -- 172.21.15.37:0/39052118 shutdown_connections 2024-09-17T04:33:56.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.220+0000 7fd3a6d50640 1 -- 172.21.15.37:0/39052118 wait complete. 2024-09-17T04:33:56.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.222+0000 7fd3a6d50640 1 Processor -- start 2024-09-17T04:33:56.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.222+0000 7fd3a6d50640 1 -- start start 2024-09-17T04:33:56.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.223+0000 7fd3a6d50640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3a00cef40 0x7fd3a0159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:56.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.223+0000 7fd3a6d50640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3a015c020 0x7fd3a01f7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:56.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.223+0000 7fd3a6d50640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd3a01f7ac0 0x7fd3a01f8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:56.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.223+0000 7fd3a6d50640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a00b8060 con 0x7fd3a00cef40 2024-09-17T04:33:56.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.223+0000 7fd3a6d50640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a00b7ee0 con 0x7fd3a015c020 2024-09-17T04:33:56.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.223+0000 7fd3a6d50640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fd3a00b81e0 con 0x7fd3a01f7ac0 2024-09-17T04:33:56.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.223+0000 7fd3a4aec640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3a00cef40 0x7fd3a0159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:56.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.223+0000 7fd3a4aec640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3a00cef40 0x7fd3a0159be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34790/0 (socket says 172.21.15.37:34790) 2024-09-17T04:33:56.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.223+0000 7fd3a4aec640 1 -- 172.21.15.37:0/914316611 learned_addr learned my addr 172.21.15.37:0/914316611 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:56.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.223+0000 7fd3a52ed640 1 --2- 172.21.15.37:0/914316611 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd3a01f7ac0 0x7fd3a01f8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:56.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.223+0000 7fd397fff640 1 --2- 172.21.15.37:0/914316611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3a015c020 0x7fd3a01f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:56.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.224+0000 7fd3a4aec640 1 -- 172.21.15.37:0/914316611 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd3a01f7ac0 msgr2=0x7fd3a01f8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:56.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.224+0000 7fd3a4aec640 1 --2- 172.21.15.37:0/914316611 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd3a01f7ac0 0x7fd3a01f8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.224+0000 7fd3a4aec640 1 -- 172.21.15.37:0/914316611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3a015c020 msgr2=0x7fd3a01f7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:56.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.224+0000 7fd3a4aec640 1 --2- 172.21.15.37:0/914316611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3a015c020 0x7fd3a01f7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.224+0000 7fd3a4aec640 1 -- 172.21.15.37:0/914316611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a015c800 con 0x7fd3a00cef40 2024-09-17T04:33:56.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.224+0000 7fd3a52ed640 1 --2- 172.21.15.37:0/914316611 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd3a01f7ac0 0x7fd3a01f8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:56.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.224+0000 7fd397fff640 1 --2- 172.21.15.37:0/914316611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3a015c020 0x7fd3a01f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:56.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.224+0000 7fd3a4aec640 1 --2- 172.21.15.37:0/914316611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3a00cef40 0x7fd3a0159be0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00c490 tx=0x7fd39c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:56.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.224+0000 7fd3957fa640 1 -- 172.21.15.37:0/914316611 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c011790 con 0x7fd3a00cef40 2024-09-17T04:33:56.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.224+0000 7fd3957fa640 1 -- 172.21.15.37:0/914316611 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd39c017070 con 0x7fd3a00cef40 2024-09-17T04:33:56.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.224+0000 7fd3a6d50640 1 -- 172.21.15.37:0/914316611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a015cae0 con 0x7fd3a00cef40 2024-09-17T04:33:56.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.224+0000 7fd3957fa640 1 -- 172.21.15.37:0/914316611 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c011ce0 con 0x7fd3a00cef40 2024-09-17T04:33:56.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.224+0000 7fd3a6d50640 1 -- 172.21.15.37:0/914316611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a00cc830 con 0x7fd3a00cef40 2024-09-17T04:33:56.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.225+0000 7fd3a6d50640 1 -- 172.21.15.37:0/914316611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3a00d3fe0 con 0x7fd3a00cef40 2024-09-17T04:33:56.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.226+0000 7fd3957fa640 1 -- 172.21.15.37:0/914316611 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd39c01c460 con 0x7fd3a00cef40 2024-09-17T04:33:56.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.227+0000 7fd3957fa640 1 --2- 172.21.15.37:0/914316611 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd38c078bc0 0x7fd38c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:56.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.227+0000 7fd3957fa640 1 -- 172.21.15.37:0/914316611 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fd39c09a830 con 0x7fd3a00cef40 2024-09-17T04:33:56.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.227+0000 7fd397fff640 1 --2- 172.21.15.37:0/914316611 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd38c078bc0 0x7fd38c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:56.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.228+0000 7fd397fff640 1 --2- 172.21.15.37:0/914316611 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd38c078bc0 0x7fd38c07b0d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd390034000 tx=0x7fd390004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:56.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.232+0000 7fd3957fa640 1 -- 172.21.15.37:0/914316611 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd39c062f40 con 0x7fd3a00cef40 2024-09-17T04:33:56.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.489+0000 7fd3a6d50640 1 -- 172.21.15.37:0/914316611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fd3a0161e20 con 0x7fd3a00cef40 2024-09-17T04:33:56.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.491+0000 7fd3957fa640 1 -- 172.21.15.37:0/914316611 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v15) ==== 97+0+0 (secure 0 0 0) 0x7fd39c062840 con 0x7fd3a00cef40 2024-09-17T04:33:56.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.494+0000 7fd3a6d50640 1 -- 172.21.15.37:0/914316611 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd38c078bc0 msgr2=0x7fd38c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:56.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.494+0000 7fd3a6d50640 1 --2- 172.21.15.37:0/914316611 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd38c078bc0 0x7fd38c07b0d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd390034000 tx=0x7fd390004730 comp rx=0 tx=0).stop 2024-09-17T04:33:56.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.494+0000 7fd3a6d50640 1 -- 172.21.15.37:0/914316611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3a00cef40 msgr2=0x7fd3a0159be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:56.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.494+0000 7fd3a6d50640 1 --2- 172.21.15.37:0/914316611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3a00cef40 0x7fd3a0159be0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00c490 tx=0x7fd39c00c950 comp rx=0 tx=0).stop 2024-09-17T04:33:56.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.494+0000 7fd3a6d50640 1 -- 172.21.15.37:0/914316611 shutdown_connections 2024-09-17T04:33:56.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.494+0000 7fd3a6d50640 1 --2- 172.21.15.37:0/914316611 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd38c078bc0 0x7fd38c07b0d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.494+0000 7fd3a6d50640 1 --2- 172.21.15.37:0/914316611 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd3a01f7ac0 0x7fd3a01f8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.494+0000 7fd3a6d50640 1 --2- 172.21.15.37:0/914316611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3a015c020 0x7fd3a01f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.494+0000 7fd3a6d50640 1 --2- 172.21.15.37:0/914316611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3a00cef40 0x7fd3a0159be0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.494+0000 7fd3a6d50640 1 -- 172.21.15.37:0/914316611 >> 172.21.15.37:0/914316611 conn(0x7fd3a00d44c0 msgr2=0x7fd3a015d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:56.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.494+0000 7fd3a6d50640 1 -- 172.21.15.37:0/914316611 shutdown_connections 2024-09-17T04:33:56.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.494+0000 7fd3a6d50640 1 -- 172.21.15.37:0/914316611 wait complete. 2024-09-17T04:33:56.505 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-17T04:33:56.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.623+0000 7f225b441640 1 -- 172.21.15.37:0/2656266239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22540d3fe0 msgr2=0x7f225415e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:56.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.623+0000 7f225b441640 1 --2- 172.21.15.37:0/2656266239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22540d3fe0 0x7f225415e8e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2250009830 tx=0x7f225002a510 comp rx=0 tx=0).stop 2024-09-17T04:33:56.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.623+0000 7f225b441640 1 -- 172.21.15.37:0/2656266239 shutdown_connections 2024-09-17T04:33:56.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.623+0000 7f225b441640 1 --2- 172.21.15.37:0/2656266239 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f22540cef40 0x7f22540b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.623+0000 7f225b441640 1 --2- 172.21.15.37:0/2656266239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22540d3fe0 0x7f225415e8e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.623+0000 7f225b441640 1 -- 172.21.15.37:0/2656266239 >> 172.21.15.37:0/2656266239 conn(0x7f22540d44c0 msgr2=0x7f22540d48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:56.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.623+0000 7f225b441640 1 -- 172.21.15.37:0/2656266239 shutdown_connections 2024-09-17T04:33:56.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.623+0000 7f225b441640 1 -- 172.21.15.37:0/2656266239 wait complete. 2024-09-17T04:33:56.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.625+0000 7f225b441640 1 Processor -- start 2024-09-17T04:33:56.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.625+0000 7f225b441640 1 -- start start 2024-09-17T04:33:56.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.627+0000 7f225b441640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f22540b6480 0x7f2254160170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:56.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.627+0000 7f225b441640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22540cef40 0x7f22540c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:56.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.627+0000 7f225b441640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22541e9280 0x7f22541e9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:56.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.627+0000 7f225b441640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f225406e470 con 0x7f22541e9280 2024-09-17T04:33:56.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.627+0000 7f225b441640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f22540b3d30 con 0x7f22540cef40 2024-09-17T04:33:56.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.627+0000 7f225b441640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f225406e5f0 con 0x7f22540b6480 2024-09-17T04:33:56.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.627+0000 7f22599de640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22541e9280 0x7f22541e9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:56.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.627+0000 7f22599de640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22541e9280 0x7f22541e9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34802/0 (socket says 172.21.15.37:34802) 2024-09-17T04:33:56.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.627+0000 7f22589dc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22540cef40 0x7f22540c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:56.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.627+0000 7f22599de640 1 -- 172.21.15.37:0/1812848771 learned_addr learned my addr 172.21.15.37:0/1812848771 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:56.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.627+0000 7f22591dd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f22540b6480 0x7f2254160170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:56.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.627+0000 7f22599de640 1 -- 172.21.15.37:0/1812848771 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f22540b6480 msgr2=0x7f2254160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:56.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.627+0000 7f22599de640 1 --2- 172.21.15.37:0/1812848771 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f22540b6480 0x7f2254160170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.627+0000 7f22599de640 1 -- 172.21.15.37:0/1812848771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22540cef40 msgr2=0x7f22540c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:56.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.627+0000 7f22599de640 1 --2- 172.21.15.37:0/1812848771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22540cef40 0x7f22540c96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.627+0000 7f22599de640 1 -- 172.21.15.37:0/1812848771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22541e9bd0 con 0x7f22541e9280 2024-09-17T04:33:56.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.627+0000 7f22599de640 1 --2- 172.21.15.37:0/1812848771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22541e9280 0x7f22541e9690 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f224c00cb40 tx=0x7f224c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:56.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.628+0000 7f2245ffb640 1 -- 172.21.15.37:0/1812848771 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c016020 con 0x7f22541e9280 2024-09-17T04:33:56.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.628+0000 7f225b441640 1 -- 172.21.15.37:0/1812848771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22541f89a0 con 0x7f22541e9280 2024-09-17T04:33:56.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.628+0000 7f2245ffb640 1 -- 172.21.15.37:0/1812848771 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f224c00ad90 con 0x7f22541e9280 2024-09-17T04:33:56.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.628+0000 7f2245ffb640 1 -- 172.21.15.37:0/1812848771 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c012680 con 0x7f22541e9280 2024-09-17T04:33:56.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.628+0000 7f225b441640 1 -- 172.21.15.37:0/1812848771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22541f8f60 con 0x7f22541e9280 2024-09-17T04:33:56.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.629+0000 7f225b441640 1 -- 172.21.15.37:0/1812848771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f225406eaa0 con 0x7f22541e9280 2024-09-17T04:33:56.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.630+0000 7f2245ffb640 1 -- 172.21.15.37:0/1812848771 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f224c019020 con 0x7f22541e9280 2024-09-17T04:33:56.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.630+0000 7f2245ffb640 1 --2- 172.21.15.37:0/1812848771 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f2238078bc0 0x7f223807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:56.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.630+0000 7f2245ffb640 1 -- 172.21.15.37:0/1812848771 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f224c09ad70 con 0x7f22541e9280 2024-09-17T04:33:56.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.630+0000 7f22591dd640 1 --2- 172.21.15.37:0/1812848771 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f2238078bc0 0x7f223807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:56.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.631+0000 7f22591dd640 1 --2- 172.21.15.37:0/1812848771 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f2238078bc0 0x7f223807b0d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2250034000 tx=0x7f2250004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:56.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.635+0000 7f2245ffb640 1 -- 172.21.15.37:0/1812848771 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f224c020020 con 0x7f22541e9280 2024-09-17T04:33:56.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.890+0000 7f225b441640 1 -- 172.21.15.37:0/1812848771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f22540d4970 con 0x7f22541e9280 2024-09-17T04:33:56.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.893+0000 7f2245ffb640 1 -- 172.21.15.37:0/1812848771 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v16) ==== 90+0+0 (secure 0 0 0) 0x7f22541e9bd0 con 0x7f22541e9280 2024-09-17T04:33:56.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.896+0000 7f225b441640 1 -- 172.21.15.37:0/1812848771 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f2238078bc0 msgr2=0x7f223807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:56.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.896+0000 7f225b441640 1 --2- 172.21.15.37:0/1812848771 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f2238078bc0 0x7f223807b0d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2250034000 tx=0x7f2250004730 comp rx=0 tx=0).stop 2024-09-17T04:33:56.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.896+0000 7f225b441640 1 -- 172.21.15.37:0/1812848771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22541e9280 msgr2=0x7f22541e9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:56.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.896+0000 7f225b441640 1 --2- 172.21.15.37:0/1812848771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22541e9280 0x7f22541e9690 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f224c00cb40 tx=0x7f224c00a060 comp rx=0 tx=0).stop 2024-09-17T04:33:56.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.896+0000 7f225b441640 1 -- 172.21.15.37:0/1812848771 shutdown_connections 2024-09-17T04:33:56.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.896+0000 7f225b441640 1 --2- 172.21.15.37:0/1812848771 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f2238078bc0 0x7f223807b0d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.896+0000 7f225b441640 1 --2- 172.21.15.37:0/1812848771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22541e9280 0x7f22541e9690 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.896+0000 7f225b441640 1 --2- 172.21.15.37:0/1812848771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22540cef40 0x7f22540c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.896+0000 7f225b441640 1 --2- 172.21.15.37:0/1812848771 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f22540b6480 0x7f2254160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:56.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.896+0000 7f225b441640 1 -- 172.21.15.37:0/1812848771 >> 172.21.15.37:0/1812848771 conn(0x7f22540d44c0 msgr2=0x7f225406dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:56.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.896+0000 7f225b441640 1 -- 172.21.15.37:0/1812848771 shutdown_connections 2024-09-17T04:33:56.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:56.896+0000 7f225b441640 1 -- 172.21.15.37:0/1812848771 wait complete. 2024-09-17T04:33:56.907 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T04:33:57.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.023+0000 7f23501ad640 1 -- 172.21.15.37:0/1624809196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23480d3fe0 msgr2=0x7f234815e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:57.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.023+0000 7f23501ad640 1 --2- 172.21.15.37:0/1624809196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23480d3fe0 0x7f234815e8e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f23480b38c0 tx=0x7f233c02a4a0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.023+0000 7f23501ad640 1 -- 172.21.15.37:0/1624809196 shutdown_connections 2024-09-17T04:33:57.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.023+0000 7f23501ad640 1 --2- 172.21.15.37:0/1624809196 >> v2:172.21.15.92:3300/0 conn(0x7f23480b6650 0x7f23480b7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.023+0000 7f23501ad640 1 --2- 172.21.15.37:0/1624809196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23480d3fe0 0x7f234815e8e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.023+0000 7f23501ad640 1 -- 172.21.15.37:0/1624809196 >> 172.21.15.37:0/1624809196 conn(0x7f23480d44c0 msgr2=0x7f23480d48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:57.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.024+0000 7f23501ad640 1 -- 172.21.15.37:0/1624809196 shutdown_connections 2024-09-17T04:33:57.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.024+0000 7f23501ad640 1 -- 172.21.15.37:0/1624809196 wait complete. 2024-09-17T04:33:57.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.025+0000 7f23501ad640 1 Processor -- start 2024-09-17T04:33:57.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.025+0000 7f23501ad640 1 -- start start 2024-09-17T04:33:57.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.027+0000 7f23501ad640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f23480cef40 0x7f23480b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:57.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.027+0000 7f23501ad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23481f8a50 0x7f23481f8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:57.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.027+0000 7f23501ad640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23481f93a0 0x7f23480ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:57.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.027+0000 7f23501ad640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f23480b8230 con 0x7f23481f8a50 2024-09-17T04:33:57.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.027+0000 7f23501ad640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f23480b80b0 con 0x7f23481f93a0 2024-09-17T04:33:57.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.027+0000 7f23501ad640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f23480b83b0 con 0x7f23480cef40 2024-09-17T04:33:57.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.027+0000 7f234d748640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23481f8a50 0x7f23481f8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:57.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.027+0000 7f234d748640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23481f8a50 0x7f23481f8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34820/0 (socket says 172.21.15.37:34820) 2024-09-17T04:33:57.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.027+0000 7f234d748640 1 -- 172.21.15.37:0/3730983019 learned_addr learned my addr 172.21.15.37:0/3730983019 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:57.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.027+0000 7f234e74a640 1 --2- 172.21.15.37:0/3730983019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23481f93a0 0x7f23480ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:57.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.027+0000 7f234df49640 1 --2- 172.21.15.37:0/3730983019 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f23480cef40 0x7f23480b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:57.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.027+0000 7f234d748640 1 -- 172.21.15.37:0/3730983019 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f23480cef40 msgr2=0x7f23480b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:57.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.027+0000 7f234d748640 1 --2- 172.21.15.37:0/3730983019 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f23480cef40 0x7f23480b2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.027+0000 7f234d748640 1 -- 172.21.15.37:0/3730983019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23481f93a0 msgr2=0x7f23480ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:57.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.027+0000 7f234d748640 1 --2- 172.21.15.37:0/3730983019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23481f93a0 0x7f23480ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.027+0000 7f234d748640 1 -- 172.21.15.37:0/3730983019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2348150660 con 0x7f23481f8a50 2024-09-17T04:33:57.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.027+0000 7f234d748640 1 --2- 172.21.15.37:0/3730983019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23481f8a50 0x7f23481f8e60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f234400c390 tx=0x7f234400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:57.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.027+0000 7f233a7fc640 1 -- 172.21.15.37:0/3730983019 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2344017070 con 0x7f23481f8a50 2024-09-17T04:33:57.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.027+0000 7f233a7fc640 1 -- 172.21.15.37:0/3730983019 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23440035f0 con 0x7f23481f8a50 2024-09-17T04:33:57.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.027+0000 7f23501ad640 1 -- 172.21.15.37:0/3730983019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f234814fdc0 con 0x7f23481f8a50 2024-09-17T04:33:57.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.027+0000 7f233a7fc640 1 -- 172.21.15.37:0/3730983019 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2344003a20 con 0x7f23481f8a50 2024-09-17T04:33:57.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.028+0000 7f23501ad640 1 -- 172.21.15.37:0/3730983019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2348150380 con 0x7f23481f8a50 2024-09-17T04:33:57.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.028+0000 7f23501ad640 1 -- 172.21.15.37:0/3730983019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23480b8860 con 0x7f23481f8a50 2024-09-17T04:33:57.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.029+0000 7f233a7fc640 1 -- 172.21.15.37:0/3730983019 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f234401a070 con 0x7f23481f8a50 2024-09-17T04:33:57.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.030+0000 7f233a7fc640 1 --2- 172.21.15.37:0/3730983019 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f232c078b90 0x7f232c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:57.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.030+0000 7f233a7fc640 1 -- 172.21.15.37:0/3730983019 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f2344099c70 con 0x7f23481f8a50 2024-09-17T04:33:57.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.030+0000 7f234df49640 1 --2- 172.21.15.37:0/3730983019 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f232c078b90 0x7f232c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:57.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.031+0000 7f234df49640 1 --2- 172.21.15.37:0/3730983019 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f232c078b90 0x7f232c07b0a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f233c034000 tx=0x7f233c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:57.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.034+0000 7f233a7fc640 1 -- 172.21.15.37:0/3730983019 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f2344062490 con 0x7f23481f8a50 2024-09-17T04:33:57.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.221+0000 7f23501ad640 1 -- 172.21.15.37:0/3730983019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2348162180 con 0x7f23481f8a50 2024-09-17T04:33:57.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.222+0000 7f233a7fc640 1 -- 172.21.15.37:0/3730983019 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) ==== 84+0+65 (secure 0 0 0) 0x7f2344061d90 con 0x7f23481f8a50 2024-09-17T04:33:57.222 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:33:57.222 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-09-17T04:33:57.222 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T04:33:57.222 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-09-17T04:33:57.222 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-09-17T04:33:57.222 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:33:57.222 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-09-17T04:33:57.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.224+0000 7f23501ad640 1 -- 172.21.15.37:0/3730983019 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f232c078b90 msgr2=0x7f232c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:57.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.224+0000 7f23501ad640 1 --2- 172.21.15.37:0/3730983019 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f232c078b90 0x7f232c07b0a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f233c034000 tx=0x7f233c004730 comp rx=0 tx=0).stop 2024-09-17T04:33:57.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.224+0000 7f23501ad640 1 -- 172.21.15.37:0/3730983019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23481f8a50 msgr2=0x7f23481f8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:57.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.224+0000 7f23501ad640 1 --2- 172.21.15.37:0/3730983019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23481f8a50 0x7f23481f8e60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f234400c390 tx=0x7f234400c850 comp rx=0 tx=0).stop 2024-09-17T04:33:57.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.224+0000 7f23501ad640 1 -- 172.21.15.37:0/3730983019 shutdown_connections 2024-09-17T04:33:57.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.224+0000 7f23501ad640 1 --2- 172.21.15.37:0/3730983019 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f232c078b90 0x7f232c07b0a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.224+0000 7f23501ad640 1 --2- 172.21.15.37:0/3730983019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23481f93a0 0x7f23480ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.224+0000 7f23501ad640 1 --2- 172.21.15.37:0/3730983019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23481f8a50 0x7f23481f8e60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.224+0000 7f23501ad640 1 --2- 172.21.15.37:0/3730983019 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f23480cef40 0x7f23480b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.224+0000 7f23501ad640 1 -- 172.21.15.37:0/3730983019 >> 172.21.15.37:0/3730983019 conn(0x7f23480d44c0 msgr2=0x7f234815d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:57.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.224+0000 7f23501ad640 1 -- 172.21.15.37:0/3730983019 shutdown_connections 2024-09-17T04:33:57.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.224+0000 7f23501ad640 1 -- 172.21.15.37:0/3730983019 wait complete. 2024-09-17T04:33:57.236 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T04:33:57.236 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T04:33:57.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.357+0000 7ff476b82640 1 -- 172.21.15.37:0/1157058063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4700b4050 msgr2=0x7ff4700d0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:57.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.357+0000 7ff476b82640 1 --2- 172.21.15.37:0/1157058063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4700b4050 0x7ff4700d0cb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff468005fd0 tx=0x7ff46802a820 comp rx=0 tx=0).stop 2024-09-17T04:33:57.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.357+0000 7ff476b82640 1 -- 172.21.15.37:0/1157058063 shutdown_connections 2024-09-17T04:33:57.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.357+0000 7ff476b82640 1 --2- 172.21.15.37:0/1157058063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4700b4050 0x7ff4700d0cb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.357+0000 7ff476b82640 1 --2- 172.21.15.37:0/1157058063 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff4700b9630 0x7ff4700b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.357+0000 7ff476b82640 1 -- 172.21.15.37:0/1157058063 >> 172.21.15.37:0/1157058063 conn(0x7ff47009d6f0 msgr2=0x7ff4700d4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:57.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.357+0000 7ff476b82640 1 -- 172.21.15.37:0/1157058063 shutdown_connections 2024-09-17T04:33:57.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.357+0000 7ff476b82640 1 -- 172.21.15.37:0/1157058063 wait complete. 2024-09-17T04:33:57.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.359+0000 7ff476b82640 1 Processor -- start 2024-09-17T04:33:57.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.359+0000 7ff476b82640 1 -- start start 2024-09-17T04:33:57.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.360+0000 7ff476b82640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4700b4050 0x7ff4701f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:57.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.360+0000 7ff476b82640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4700b9630 0x7ff470163e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:57.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.360+0000 7ff476b82640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff4700d1af0 0x7ff4701f8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:57.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.360+0000 7ff476b82640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff4700d1370 con 0x7ff4700b4050 2024-09-17T04:33:57.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.360+0000 7ff476b82640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff4700d11f0 con 0x7ff4700b9630 2024-09-17T04:33:57.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.360+0000 7ff476b82640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7ff4700d14f0 con 0x7ff4700d1af0 2024-09-17T04:33:57.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.360+0000 7ff47491e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4700b4050 0x7ff4701f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:57.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.361+0000 7ff47491e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4700b4050 0x7ff4701f53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34822/0 (socket says 172.21.15.37:34822) 2024-09-17T04:33:57.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.361+0000 7ff47491e640 1 -- 172.21.15.37:0/2581788170 learned_addr learned my addr 172.21.15.37:0/2581788170 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:57.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.361+0000 7ff46ffff640 1 --2- 172.21.15.37:0/2581788170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4700b9630 0x7ff470163e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:57.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.361+0000 7ff47511f640 1 --2- 172.21.15.37:0/2581788170 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff4700d1af0 0x7ff4701f8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:57.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.361+0000 7ff47491e640 1 -- 172.21.15.37:0/2581788170 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff4700d1af0 msgr2=0x7ff4701f8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:57.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.361+0000 7ff47491e640 1 --2- 172.21.15.37:0/2581788170 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff4700d1af0 0x7ff4701f8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.361+0000 7ff47491e640 1 -- 172.21.15.37:0/2581788170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4700b9630 msgr2=0x7ff470163e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:57.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.361+0000 7ff47491e640 1 --2- 172.21.15.37:0/2581788170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4700b9630 0x7ff470163e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:57.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.361+0000 7ff47491e640 1 -- 172.21.15.37:0/2581788170 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4701667e0 con 0x7ff4700b4050 2024-09-17T04:33:57.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.361+0000 7ff47491e640 1 --2- 172.21.15.37:0/2581788170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4700b4050 0x7ff4701f53a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff46400c3f0 tx=0x7ff46400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:57.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.361+0000 7ff46d7fa640 1 -- 172.21.15.37:0/2581788170 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff464017070 con 0x7ff4700b4050 2024-09-17T04:33:57.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.361+0000 7ff46d7fa640 1 -- 172.21.15.37:0/2581788170 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff464003640 con 0x7ff4700b4050 2024-09-17T04:33:57.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.361+0000 7ff476b82640 1 -- 172.21.15.37:0/2581788170 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff470165f40 con 0x7ff4700b4050 2024-09-17T04:33:57.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.361+0000 7ff46d7fa640 1 -- 172.21.15.37:0/2581788170 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff464003a90 con 0x7ff4700b4050 2024-09-17T04:33:57.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.361+0000 7ff476b82640 1 -- 172.21.15.37:0/2581788170 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff470166500 con 0x7ff4700b4050 2024-09-17T04:33:57.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.362+0000 7ff476b82640 1 -- 172.21.15.37:0/2581788170 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4700d0730 con 0x7ff4700b4050 2024-09-17T04:33:57.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.363+0000 7ff46d7fa640 1 -- 172.21.15.37:0/2581788170 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff46401a070 con 0x7ff4700b4050 2024-09-17T04:33:57.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.364+0000 7ff46d7fa640 1 --2- 172.21.15.37:0/2581788170 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff454078b90 0x7ff45407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:57.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.364+0000 7ff46d7fa640 1 -- 172.21.15.37:0/2581788170 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7ff464099b10 con 0x7ff4700b4050 2024-09-17T04:33:57.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.364+0000 7ff46ffff640 1 --2- 172.21.15.37:0/2581788170 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff454078b90 0x7ff45407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:57.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.365+0000 7ff46ffff640 1 --2- 172.21.15.37:0/2581788170 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff454078b90 0x7ff45407b0a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff4701f6480 tx=0x7ff45c005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:57.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.368+0000 7ff46d7fa640 1 -- 172.21.15.37:0/2581788170 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff4700b4460 con 0x7ff4700b4050 2024-09-17T04:33:57.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:57.554+0000 7ff476b82640 1 -- 172.21.15.37:0/2581788170 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff4700d0c60 con 0x7ff4700b4050 2024-09-17T04:33:58.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.091+0000 7ff46d7fa640 1 -- 172.21.15.37:0/2581788170 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) ==== 138+0+0 (secure 0 0 0) 0x7ff464061c60 con 0x7ff4700b4050 2024-09-17T04:33:58.091 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-09-17T04:33:58.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.095+0000 7ff476b82640 1 -- 172.21.15.37:0/2581788170 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff454078b90 msgr2=0x7ff45407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:58.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.095+0000 7ff476b82640 1 --2- 172.21.15.37:0/2581788170 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff454078b90 0x7ff45407b0a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff4701f6480 tx=0x7ff45c005f90 comp rx=0 tx=0).stop 2024-09-17T04:33:58.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.095+0000 7ff476b82640 1 -- 172.21.15.37:0/2581788170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4700b4050 msgr2=0x7ff4701f53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:58.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.095+0000 7ff476b82640 1 --2- 172.21.15.37:0/2581788170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4700b4050 0x7ff4701f53a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff46400c3f0 tx=0x7ff46400c8b0 comp rx=0 tx=0).stop 2024-09-17T04:33:58.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.097+0000 7ff476b82640 1 -- 172.21.15.37:0/2581788170 shutdown_connections 2024-09-17T04:33:58.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.097+0000 7ff476b82640 1 --2- 172.21.15.37:0/2581788170 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff454078b90 0x7ff45407b0a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:58.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.097+0000 7ff476b82640 1 --2- 172.21.15.37:0/2581788170 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff4700d1af0 0x7ff4701f8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:58.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.097+0000 7ff476b82640 1 --2- 172.21.15.37:0/2581788170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4700b9630 0x7ff470163e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:58.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.097+0000 7ff476b82640 1 --2- 172.21.15.37:0/2581788170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4700b4050 0x7ff4701f53a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:58.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.097+0000 7ff476b82640 1 -- 172.21.15.37:0/2581788170 >> 172.21.15.37:0/2581788170 conn(0x7ff47009d6f0 msgr2=0x7ff4700b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:58.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.098+0000 7ff476b82640 1 -- 172.21.15.37:0/2581788170 shutdown_connections 2024-09-17T04:33:58.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.099+0000 7ff476b82640 1 -- 172.21.15.37:0/2581788170 wait complete. 2024-09-17T04:33:58.114 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T04:33:58.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.245+0000 7f08df96a640 1 -- 172.21.15.37:0/3253546797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08d80b34e0 msgr2=0x7f08d80d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:58.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.245+0000 7f08df96a640 1 --2- 172.21.15.37:0/3253546797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08d80b34e0 0x7f08d80d0f20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009830 tx=0x7f08cc02a530 comp rx=0 tx=0).stop 2024-09-17T04:33:58.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.245+0000 7f08df96a640 1 -- 172.21.15.37:0/3253546797 shutdown_connections 2024-09-17T04:33:58.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.245+0000 7f08df96a640 1 --2- 172.21.15.37:0/3253546797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08d80b34e0 0x7f08d80d0f20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:58.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.245+0000 7f08df96a640 1 -- 172.21.15.37:0/3253546797 >> 172.21.15.37:0/3253546797 conn(0x7f08d808bfb0 msgr2=0x7f08d809d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:58.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.245+0000 7f08df96a640 1 -- 172.21.15.37:0/3253546797 shutdown_connections 2024-09-17T04:33:58.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.245+0000 7f08df96a640 1 -- 172.21.15.37:0/3253546797 wait complete. 2024-09-17T04:33:58.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.247+0000 7f08df96a640 1 Processor -- start 2024-09-17T04:33:58.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.247+0000 7f08df96a640 1 -- start start 2024-09-17T04:33:58.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.248+0000 7f08df96a640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f08d80b34e0 0x7f08d8156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:58.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.248+0000 7f08df96a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08d80d1460 0x7f08d80cd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:58.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.248+0000 7f08df96a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08d80caa70 0x7f08d80cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:58.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.248+0000 7f08df96a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f08d80cfad0 con 0x7f08d80d1460 2024-09-17T04:33:58.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.248+0000 7f08df96a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f08d80cf950 con 0x7f08d80caa70 2024-09-17T04:33:58.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.248+0000 7f08df96a640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f08d80cfc50 con 0x7f08d80b34e0 2024-09-17T04:33:58.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.248+0000 7f08dcf05640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08d80d1460 0x7f08d80cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:58.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.249+0000 7f08dcf05640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08d80d1460 0x7f08d80cd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34844/0 (socket says 172.21.15.37:34844) 2024-09-17T04:33:58.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.249+0000 7f08dcf05640 1 -- 172.21.15.37:0/1147971982 learned_addr learned my addr 172.21.15.37:0/1147971982 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:58.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.249+0000 7f08dd706640 1 --2- 172.21.15.37:0/1147971982 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f08d80b34e0 0x7f08d8156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:58.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.249+0000 7f08ddf07640 1 --2- 172.21.15.37:0/1147971982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08d80caa70 0x7f08d80cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:58.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.249+0000 7f08dcf05640 1 -- 172.21.15.37:0/1147971982 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f08d80b34e0 msgr2=0x7f08d8156d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:58.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.249+0000 7f08dcf05640 1 --2- 172.21.15.37:0/1147971982 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f08d80b34e0 0x7f08d8156d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:58.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.249+0000 7f08dcf05640 1 -- 172.21.15.37:0/1147971982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08d80caa70 msgr2=0x7f08d80cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:58.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.249+0000 7f08dcf05640 1 --2- 172.21.15.37:0/1147971982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08d80caa70 0x7f08d80cddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:58.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.249+0000 7f08dcf05640 1 -- 172.21.15.37:0/1147971982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d80ce650 con 0x7f08d80d1460 2024-09-17T04:33:58.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.249+0000 7f08ddf07640 1 --2- 172.21.15.37:0/1147971982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08d80caa70 0x7f08d80cddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:33:58.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.249+0000 7f08dcf05640 1 --2- 172.21.15.37:0/1147971982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08d80d1460 0x7f08d80cd890 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009950 tx=0x7f08cc033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:58.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.249+0000 7f08c9ffb640 1 -- 172.21.15.37:0/1147971982 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc031d60 con 0x7f08d80d1460 2024-09-17T04:33:58.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.249+0000 7f08c9ffb640 1 -- 172.21.15.37:0/1147971982 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08cc0330d0 con 0x7f08d80d1460 2024-09-17T04:33:58.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.249+0000 7f08df96a640 1 -- 172.21.15.37:0/1147971982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d815ef80 con 0x7f08d80d1460 2024-09-17T04:33:58.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.249+0000 7f08c9ffb640 1 -- 172.21.15.37:0/1147971982 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc033520 con 0x7f08d80d1460 2024-09-17T04:33:58.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.249+0000 7f08df96a640 1 -- 172.21.15.37:0/1147971982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d815f540 con 0x7f08d80d1460 2024-09-17T04:33:58.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.251+0000 7f08df96a640 1 -- 172.21.15.37:0/1147971982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08d80b9ca0 con 0x7f08d80d1460 2024-09-17T04:33:58.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.251+0000 7f08c9ffb640 1 -- 172.21.15.37:0/1147971982 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f08cc0035f0 con 0x7f08d80d1460 2024-09-17T04:33:58.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.252+0000 7f08c9ffb640 1 --2- 172.21.15.37:0/1147971982 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f08bc078bc0 0x7f08bc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:58.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.252+0000 7f08c9ffb640 1 -- 172.21.15.37:0/1147971982 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5227+0+0 (secure 0 0 0) 0x7f08cc0bc160 con 0x7f08d80d1460 2024-09-17T04:33:58.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.252+0000 7f08dd706640 1 --2- 172.21.15.37:0/1147971982 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f08bc078bc0 0x7f08bc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:58.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.253+0000 7f08dd706640 1 --2- 172.21.15.37:0/1147971982 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f08bc078bc0 0x7f08bc07b0d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f08d4001090 tx=0x7f08d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:58.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.257+0000 7f08c9ffb640 1 -- 172.21.15.37:0/1147971982 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f08d80ce650 con 0x7f08d80d1460 2024-09-17T04:33:58.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:58.442+0000 7f08df96a640 1 -- 172.21.15.37:0/1147971982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f08d80d2500 con 0x7f08d80d1460 2024-09-17T04:33:59.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.094+0000 7f08c9ffb640 1 -- 172.21.15.37:0/1147971982 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) ==== 170+0+0 (secure 0 0 0) 0x7f08cc03f030 con 0x7f08d80d1460 2024-09-17T04:33:59.094 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-09-17T04:33:59.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.097+0000 7f08df96a640 1 -- 172.21.15.37:0/1147971982 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f08bc078bc0 msgr2=0x7f08bc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:59.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.097+0000 7f08df96a640 1 --2- 172.21.15.37:0/1147971982 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f08bc078bc0 0x7f08bc07b0d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f08d4001090 tx=0x7f08d4009040 comp rx=0 tx=0).stop 2024-09-17T04:33:59.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.097+0000 7f08df96a640 1 -- 172.21.15.37:0/1147971982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08d80d1460 msgr2=0x7f08d80cd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:59.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.097+0000 7f08df96a640 1 --2- 172.21.15.37:0/1147971982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08d80d1460 0x7f08d80cd890 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009950 tx=0x7f08cc033d60 comp rx=0 tx=0).stop 2024-09-17T04:33:59.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.097+0000 7f08df96a640 1 -- 172.21.15.37:0/1147971982 shutdown_connections 2024-09-17T04:33:59.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.097+0000 7f08df96a640 1 --2- 172.21.15.37:0/1147971982 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f08bc078bc0 0x7f08bc07b0d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.097+0000 7f08df96a640 1 --2- 172.21.15.37:0/1147971982 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08d80caa70 0x7f08d80cddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.097+0000 7f08df96a640 1 --2- 172.21.15.37:0/1147971982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f08d80d1460 0x7f08d80cd890 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.097+0000 7f08df96a640 1 --2- 172.21.15.37:0/1147971982 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f08d80b34e0 0x7f08d8156d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.097+0000 7f08df96a640 1 -- 172.21.15.37:0/1147971982 >> 172.21.15.37:0/1147971982 conn(0x7f08d808bfb0 msgr2=0x7f08d806eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:59.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.098+0000 7f08df96a640 1 -- 172.21.15.37:0/1147971982 shutdown_connections 2024-09-17T04:33:59.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.098+0000 7f08df96a640 1 -- 172.21.15.37:0/1147971982 wait complete. 2024-09-17T04:33:59.125 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T04:33:59.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.251+0000 7f15f1c9b640 1 -- 172.21.15.37:0/1488814562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15ec1608e0 msgr2=0x7f15ec0ccfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:59.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.251+0000 7f15f1c9b640 1 --2- 172.21.15.37:0/1488814562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15ec1608e0 0x7f15ec0ccfb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f15ec1612d0 tx=0x7f15d802a5c0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.251+0000 7f15f1c9b640 1 -- 172.21.15.37:0/1488814562 shutdown_connections 2024-09-17T04:33:59.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.251+0000 7f15f1c9b640 1 --2- 172.21.15.37:0/1488814562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15ec1608e0 0x7f15ec0ccfb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.251+0000 7f15f1c9b640 1 -- 172.21.15.37:0/1488814562 >> 172.21.15.37:0/1488814562 conn(0x7f15ec070ff0 msgr2=0x7f15ec071400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:33:59.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.251+0000 7f15f1c9b640 1 -- 172.21.15.37:0/1488814562 shutdown_connections 2024-09-17T04:33:59.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.251+0000 7f15f1c9b640 1 -- 172.21.15.37:0/1488814562 wait complete. 2024-09-17T04:33:59.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.253+0000 7f15f1c9b640 1 Processor -- start 2024-09-17T04:33:59.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.253+0000 7f15f1c9b640 1 -- start start 2024-09-17T04:33:59.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.254+0000 7f15f1c9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15ec0ce950 0x7f15ec1495a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:59.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.254+0000 7f15f1c9b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f15ec15d860 0x7f15ec155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:59.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.254+0000 7f15f1c9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15ec1608e0 0x7f15ec1561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:59.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.254+0000 7f15f1c9b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec1534e0 con 0x7f15ec0ce950 2024-09-17T04:33:59.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.254+0000 7f15f1c9b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec153360 con 0x7f15ec1608e0 2024-09-17T04:33:59.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.254+0000 7f15f1c9b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f15ec153660 con 0x7f15ec15d860 2024-09-17T04:33:59.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.255+0000 7f15eb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15ec0ce950 0x7f15ec1495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:59.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.255+0000 7f15eb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15ec0ce950 0x7f15ec1495a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34856/0 (socket says 172.21.15.37:34856) 2024-09-17T04:33:59.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.255+0000 7f15eb7fe640 1 -- 172.21.15.37:0/3051238392 learned_addr learned my addr 172.21.15.37:0/3051238392 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:33:59.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.255+0000 7f15ebfff640 1 --2- 172.21.15.37:0/3051238392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15ec1608e0 0x7f15ec1561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:59.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.255+0000 7f15eaffd640 1 --2- 172.21.15.37:0/3051238392 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f15ec15d860 0x7f15ec155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:59.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.255+0000 7f15eb7fe640 1 -- 172.21.15.37:0/3051238392 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f15ec15d860 msgr2=0x7f15ec155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:59.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.255+0000 7f15eb7fe640 1 --2- 172.21.15.37:0/3051238392 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f15ec15d860 0x7f15ec155c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.255+0000 7f15eb7fe640 1 -- 172.21.15.37:0/3051238392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15ec1608e0 msgr2=0x7f15ec1561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:33:59.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.255+0000 7f15eb7fe640 1 --2- 172.21.15.37:0/3051238392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15ec1608e0 0x7f15ec1561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:33:59.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.255+0000 7f15eb7fe640 1 -- 172.21.15.37:0/3051238392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15ec159b30 con 0x7f15ec0ce950 2024-09-17T04:33:59.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.255+0000 7f15eb7fe640 1 --2- 172.21.15.37:0/3051238392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15ec0ce950 0x7f15ec1495a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00c400 tx=0x7f15dc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:59.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.255+0000 7f15e3fff640 1 -- 172.21.15.37:0/3051238392 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc017070 con 0x7f15ec0ce950 2024-09-17T04:33:59.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.255+0000 7f15e3fff640 1 -- 172.21.15.37:0/3051238392 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15dc0053c0 con 0x7f15ec0ce950 2024-09-17T04:33:59.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.255+0000 7f15f1c9b640 1 -- 172.21.15.37:0/3051238392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15ec1579a0 con 0x7f15ec0ce950 2024-09-17T04:33:59.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.255+0000 7f15e3fff640 1 -- 172.21.15.37:0/3051238392 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc003850 con 0x7f15ec0ce950 2024-09-17T04:33:59.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.255+0000 7f15f1c9b640 1 -- 172.21.15.37:0/3051238392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15ec157f60 con 0x7f15ec0ce950 2024-09-17T04:33:59.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.257+0000 7f15f1c9b640 1 -- 172.21.15.37:0/3051238392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15ec0d0e90 con 0x7f15ec0ce950 2024-09-17T04:33:59.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.257+0000 7f15e3fff640 1 -- 172.21.15.37:0/3051238392 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f15dc01a070 con 0x7f15ec0ce950 2024-09-17T04:33:59.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.258+0000 7f15e3fff640 1 --2- 172.21.15.37:0/3051238392 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f15d4078bc0 0x7f15d407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:33:59.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.258+0000 7f15e3fff640 1 -- 172.21.15.37:0/3051238392 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5609+0+0 (secure 0 0 0) 0x7f15dc09a0c0 con 0x7f15ec0ce950 2024-09-17T04:33:59.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.258+0000 7f15eaffd640 1 --2- 172.21.15.37:0/3051238392 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f15d4078bc0 0x7f15d407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:33:59.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.259+0000 7f15eaffd640 1 --2- 172.21.15.37:0/3051238392 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f15d4078bc0 0x7f15d407b0d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f15d8034000 tx=0x7f15d8004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:33:59.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.263+0000 7f15e3fff640 1 -- 172.21.15.37:0/3051238392 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f15ec0d0e90 con 0x7f15ec0ce950 2024-09-17T04:33:59.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:33:59.489+0000 7f15f1c9b640 1 -- 172.21.15.37:0/3051238392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f15ec071930 con 0x7f15ec0ce950 2024-09-17T04:34:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.115+0000 7f15e3fff640 1 -- 172.21.15.37:0/3051238392 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7f15dc061e80 con 0x7f15ec0ce950 2024-09-17T04:34:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 4 and data pool 5 2024-09-17T04:34:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.123+0000 7f15f1c9b640 1 -- 172.21.15.37:0/3051238392 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f15d4078bc0 msgr2=0x7f15d407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.123+0000 7f15f1c9b640 1 --2- 172.21.15.37:0/3051238392 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f15d4078bc0 0x7f15d407b0d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f15d8034000 tx=0x7f15d8004730 comp rx=0 tx=0).stop 2024-09-17T04:34:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.123+0000 7f15f1c9b640 1 -- 172.21.15.37:0/3051238392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15ec0ce950 msgr2=0x7f15ec1495a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.123+0000 7f15f1c9b640 1 --2- 172.21.15.37:0/3051238392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15ec0ce950 0x7f15ec1495a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00c400 tx=0x7f15dc00c8c0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.124+0000 7f15f1c9b640 1 -- 172.21.15.37:0/3051238392 shutdown_connections 2024-09-17T04:34:00.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.124+0000 7f15f1c9b640 1 --2- 172.21.15.37:0/3051238392 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f15d4078bc0 0x7f15d407b0d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.124+0000 7f15f1c9b640 1 --2- 172.21.15.37:0/3051238392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15ec1608e0 0x7f15ec1561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.124+0000 7f15f1c9b640 1 --2- 172.21.15.37:0/3051238392 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f15ec15d860 0x7f15ec155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.124+0000 7f15f1c9b640 1 --2- 172.21.15.37:0/3051238392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15ec0ce950 0x7f15ec1495a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.124+0000 7f15f1c9b640 1 -- 172.21.15.37:0/3051238392 >> 172.21.15.37:0/3051238392 conn(0x7f15ec070ff0 msgr2=0x7f15ec0b3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.132+0000 7f15f1c9b640 1 -- 172.21.15.37:0/3051238392 shutdown_connections 2024-09-17T04:34:00.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.136+0000 7f15f1c9b640 1 -- 172.21.15.37:0/3051238392 wait complete. 2024-09-17T04:34:00.163 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:34:00.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.284+0000 7fe67a18b640 1 -- 172.21.15.37:0/3029556629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe67415c630 msgr2=0x7fe6740cff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:00.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.284+0000 7fe67a18b640 1 --2- 172.21.15.37:0/3029556629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe67415c630 0x7fe6740cff30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe6740b18f0 tx=0x7fe66402a4a0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.284+0000 7fe67a18b640 1 -- 172.21.15.37:0/3029556629 shutdown_connections 2024-09-17T04:34:00.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.284+0000 7fe67a18b640 1 --2- 172.21.15.37:0/3029556629 >> v2:172.21.15.92:3300/0 conn(0x7fe6740d19c0 0x7fe6740d1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.284+0000 7fe67a18b640 1 --2- 172.21.15.37:0/3029556629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe67415c630 0x7fe6740cff30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.284+0000 7fe67a18b640 1 -- 172.21.15.37:0/3029556629 >> 172.21.15.37:0/3029556629 conn(0x7fe6740cede0 msgr2=0x7fe6740cf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:00.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.285+0000 7fe67a18b640 1 -- 172.21.15.37:0/3029556629 shutdown_connections 2024-09-17T04:34:00.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.285+0000 7fe67a18b640 1 -- 172.21.15.37:0/3029556629 wait complete. 2024-09-17T04:34:00.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.286+0000 7fe67a18b640 1 Processor -- start 2024-09-17T04:34:00.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.287+0000 7fe67a18b640 1 -- start start 2024-09-17T04:34:00.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.288+0000 7fe67a18b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe6740ccbc0 0x7fe6741524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:00.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.288+0000 7fe67a18b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6740d19c0 0x7fe6740b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:00.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.288+0000 7fe67a18b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe67415c630 0x7fe6741f2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:00.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.288+0000 7fe67a18b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe67415d280 con 0x7fe67415c630 2024-09-17T04:34:00.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.288+0000 7fe67a18b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe67415d100 con 0x7fe6740d19c0 2024-09-17T04:34:00.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.288+0000 7fe67a18b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fe67415d400 con 0x7fe6740ccbc0 2024-09-17T04:34:00.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.288+0000 7fe673fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe67415c630 0x7fe6741f2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:00.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.288+0000 7fe673fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe67415c630 0x7fe6741f2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34870/0 (socket says 172.21.15.37:34870) 2024-09-17T04:34:00.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.288+0000 7fe673fff640 1 -- 172.21.15.37:0/1018780188 learned_addr learned my addr 172.21.15.37:0/1018780188 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:00.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.288+0000 7fe672ffd640 1 --2- 172.21.15.37:0/1018780188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6740d19c0 0x7fe6740b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:00.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.288+0000 7fe6737fe640 1 --2- 172.21.15.37:0/1018780188 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe6740ccbc0 0x7fe6741524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:00.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.288+0000 7fe673fff640 1 -- 172.21.15.37:0/1018780188 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe6740ccbc0 msgr2=0x7fe6741524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:00.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.288+0000 7fe673fff640 1 --2- 172.21.15.37:0/1018780188 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe6740ccbc0 0x7fe6741524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.288+0000 7fe673fff640 1 -- 172.21.15.37:0/1018780188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6740d19c0 msgr2=0x7fe6740b7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:00.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.288+0000 7fe673fff640 1 --2- 172.21.15.37:0/1018780188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6740d19c0 0x7fe6740b7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.288+0000 7fe673fff640 1 -- 172.21.15.37:0/1018780188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6740cb4d0 con 0x7fe67415c630 2024-09-17T04:34:00.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.288+0000 7fe672ffd640 1 --2- 172.21.15.37:0/1018780188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6740d19c0 0x7fe6740b7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:00.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.288+0000 7fe6737fe640 1 --2- 172.21.15.37:0/1018780188 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe6740ccbc0 0x7fe6741524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:00.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.288+0000 7fe673fff640 1 --2- 172.21.15.37:0/1018780188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe67415c630 0x7fe6741f2b10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00c6c0 tx=0x7fe66c00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:00.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.289+0000 7fe65bfff640 1 -- 172.21.15.37:0/1018780188 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c017070 con 0x7fe67415c630 2024-09-17T04:34:00.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.289+0000 7fe65bfff640 1 -- 172.21.15.37:0/1018780188 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe66c003c60 con 0x7fe67415c630 2024-09-17T04:34:00.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.289+0000 7fe67a18b640 1 -- 172.21.15.37:0/1018780188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6740cb7b0 con 0x7fe67415c630 2024-09-17T04:34:00.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.289+0000 7fe65bfff640 1 -- 172.21.15.37:0/1018780188 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c0052d0 con 0x7fe67415c630 2024-09-17T04:34:00.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.289+0000 7fe67a18b640 1 -- 172.21.15.37:0/1018780188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6740cbd70 con 0x7fe67415c630 2024-09-17T04:34:00.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.290+0000 7fe67a18b640 1 -- 172.21.15.37:0/1018780188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6740cebc0 con 0x7fe67415c630 2024-09-17T04:34:00.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.291+0000 7fe65bfff640 1 -- 172.21.15.37:0/1018780188 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe66c01a070 con 0x7fe67415c630 2024-09-17T04:34:00.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.291+0000 7fe65bfff640 1 --2- 172.21.15.37:0/1018780188 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe660078b90 0x7fe66007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:00.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.291+0000 7fe65bfff640 1 -- 172.21.15.37:0/1018780188 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fe66c09a2c0 con 0x7fe67415c630 2024-09-17T04:34:00.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.291+0000 7fe6737fe640 1 --2- 172.21.15.37:0/1018780188 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe660078b90 0x7fe66007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:00.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.292+0000 7fe6737fe640 1 --2- 172.21.15.37:0/1018780188 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe660078b90 0x7fe66007b0a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe664034000 tx=0x7fe664004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:00.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.296+0000 7fe65bfff640 1 -- 172.21.15.37:0/1018780188 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe66c062790 con 0x7fe67415c630 2024-09-17T04:34:00.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.481+0000 7fe67a18b640 1 -- 172.21.15.37:0/1018780188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe674070080 con 0x7fe67415c630 2024-09-17T04:34:00.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.482+0000 7fe65bfff640 1 -- 172.21.15.37:0/1018780188 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16682 (secure 0 0 0) 0x7fe66c062090 con 0x7fe67415c630 2024-09-17T04:34:00.482 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:00.482 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":32,"fsid":"a908a603-430c-4209-8294-0c3460b35f42","created":"2024-09-17T04:32:54.139221+0000","modified":"2024-09-17T04:33:59.490304+0000","last_up_change":"2024-09-17T04:33:06.922210+0000","last_in_change":"2024-09-17T04:32:57.754118+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:33:08.461907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:33:57.558905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:58.446650+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ea4b8d1-0d22-4dd9-b913-9df585ef24ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6809","nonce":2283587593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6811","nonce":2283587593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6815","nonce":2283587593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6813","nonce":2283587593}]},"public_addr":"172.21.15.37:6809/2283587593","cluster_addr":"172.21.15.37:6811/2283587593","heartbeat_back_addr":"172.21.15.37:6815/2283587593","heartbeat_front_addr":"172.21.15.37:6813/2283587593","state":["exists","up"]},{"osd":1,"uuid":"ac745a33-e8ad-4083-9c18-71bf81982b76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6825","nonce":2657955935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6827","nonce":2657955935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6831","nonce":2657955935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6829","nonce":2657955935}]},"public_addr":"172.21.15.37:6825/2657955935","cluster_addr":"172.21.15.37:6827/2657955935","heartbeat_back_addr":"172.21.15.37:6831/2657955935","heartbeat_front_addr":"172.21.15.37:6829/2657955935","state":["exists","up"]},{"osd":2,"uuid":"3c63f3d8-132a-4735-b0bc-d64313a46503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6817","nonce":579720176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6819","nonce":579720176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6823","nonce":579720176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6821","nonce":579720176}]},"public_addr":"172.21.15.37:6817/579720176","cluster_addr":"172.21.15.37:6819/579720176","heartbeat_back_addr":"172.21.15.37:6823/579720176","heartbeat_front_addr":"172.21.15.37:6821/579720176","state":["exists","up"]},{"osd":3,"uuid":"4f8c6ca2-ea1d-4610-9127-52c4dd8bc8e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6801","nonce":809975826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6803","nonce":809975826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6807","nonce":809975826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6805","nonce":809975826}]},"public_addr":"172.21.15.37:6801/809975826","cluster_addr":"172.21.15.37:6803/809975826","heartbeat_back_addr":"172.21.15.37:6807/809975826","heartbeat_front_addr":"172.21.15.37:6805/809975826","state":["exists","up"]},{"osd":4,"uuid":"63d0c4ab-f115-47b4-9611-12a10153473a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6819","nonce":2509785752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6821","nonce":2509785752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6825","nonce":2509785752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6823","nonce":2509785752}]},"public_addr":"172.21.15.92:6819/2509785752","cluster_addr":"172.21.15.92:6821/2509785752","heartbeat_back_addr":"172.21.15.92:6825/2509785752","heartbeat_front_addr":"172.21.15.92:6823/2509785752","state":["exists","up"]},{"osd":5,"uuid":"4c56c04d-0c07-4f76-9d75-6335121310ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6827","nonce":248263832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6829","nonce":248263832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6833","nonce":248263832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6831","nonce":248263832}]},"public_addr":"172.21.15.92:6827/248263832","cluster_addr":"172.21.15.92:6829/248263832","heartbeat_back_addr":"172.21.15.92:6833/248263832","heartbeat_front_addr":"172.21.15.92:6831/248263832","state":["exists","up"]},{"osd":6,"uuid":"b30eae7c-096a-4d24-ac51-96b8b13ecaee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6811","nonce":4078365985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6813","nonce":4078365985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6817","nonce":4078365985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6815","nonce":4078365985}]},"public_addr":"172.21.15.92:6811/4078365985","cluster_addr":"172.21.15.92:6813/4078365985","heartbeat_back_addr":"172.21.15.92:6817/4078365985","heartbeat_front_addr":"172.21.15.92:6815/4078365985","state":["exists","up"]},{"osd":7,"uuid":"62f12540-d374-4a4c-9ff7-63b7304fe77f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6803","nonce":3444787223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6805","nonce":3444787223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6809","nonce":3444787223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6807","nonce":3444787223}]},"public_addr":"172.21.15.92:6803/3444787223","cluster_addr":"172.21.15.92:6805/3444787223","heartbeat_back_addr":"172.21.15.92:6809/3444787223","heartbeat_front_addr":"172.21.15.92:6807/3444787223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.006420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.111135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.103325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.072656+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.200989+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.060455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.774773+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.865778+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:34:00.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.484+0000 7fe67a18b640 1 -- 172.21.15.37:0/1018780188 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe660078b90 msgr2=0x7fe66007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:00.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.484+0000 7fe67a18b640 1 --2- 172.21.15.37:0/1018780188 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe660078b90 0x7fe66007b0a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe664034000 tx=0x7fe664004730 comp rx=0 tx=0).stop 2024-09-17T04:34:00.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.484+0000 7fe67a18b640 1 -- 172.21.15.37:0/1018780188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe67415c630 msgr2=0x7fe6741f2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:00.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.484+0000 7fe67a18b640 1 --2- 172.21.15.37:0/1018780188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe67415c630 0x7fe6741f2b10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00c6c0 tx=0x7fe66c00cb80 comp rx=0 tx=0).stop 2024-09-17T04:34:00.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.484+0000 7fe67a18b640 1 -- 172.21.15.37:0/1018780188 shutdown_connections 2024-09-17T04:34:00.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.484+0000 7fe67a18b640 1 --2- 172.21.15.37:0/1018780188 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe660078b90 0x7fe66007b0a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.484+0000 7fe67a18b640 1 --2- 172.21.15.37:0/1018780188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe67415c630 0x7fe6741f2b10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.484+0000 7fe67a18b640 1 --2- 172.21.15.37:0/1018780188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6740d19c0 0x7fe6740b7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.484+0000 7fe67a18b640 1 --2- 172.21.15.37:0/1018780188 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe6740ccbc0 0x7fe6741524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.484+0000 7fe67a18b640 1 -- 172.21.15.37:0/1018780188 >> 172.21.15.37:0/1018780188 conn(0x7fe6740cede0 msgr2=0x7fe674072610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.485+0000 7fe67a18b640 1 -- 172.21.15.37:0/1018780188 shutdown_connections 2024-09-17T04:34:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.485+0000 7fe67a18b640 1 -- 172.21.15.37:0/1018780188 wait complete. 2024-09-17T04:34:00.496 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:34:00.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.612+0000 7fd4f2d6a640 1 -- 172.21.15.37:0/1687880228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4ec15e790 msgr2=0x7fd4ec0d0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:00.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.612+0000 7fd4f2d6a640 1 --2- 172.21.15.37:0/1687880228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4ec15e790 0x7fd4ec0d0c40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0b7770 tx=0x7fd4d802a4d0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.612+0000 7fd4f2d6a640 1 -- 172.21.15.37:0/1687880228 shutdown_connections 2024-09-17T04:34:00.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.612+0000 7fd4f2d6a640 1 --2- 172.21.15.37:0/1687880228 >> v2:172.21.15.92:3300/0 conn(0x7fd4ec15eba0 0x7fd4ec0d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.612+0000 7fd4f2d6a640 1 --2- 172.21.15.37:0/1687880228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4ec15e790 0x7fd4ec0d0c40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.612+0000 7fd4f2d6a640 1 --2- 172.21.15.37:0/1687880228 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd4ec0cfe10 0x7fd4ec162590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.612+0000 7fd4f2d6a640 1 -- 172.21.15.37:0/1687880228 >> 172.21.15.37:0/1687880228 conn(0x7fd4ec06ddb0 msgr2=0x7fd4ec06e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:00.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.612+0000 7fd4f2d6a640 1 -- 172.21.15.37:0/1687880228 shutdown_connections 2024-09-17T04:34:00.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.612+0000 7fd4f2d6a640 1 -- 172.21.15.37:0/1687880228 wait complete. 2024-09-17T04:34:00.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.614+0000 7fd4f2d6a640 1 Processor -- start 2024-09-17T04:34:00.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.614+0000 7fd4f2d6a640 1 -- start start 2024-09-17T04:34:00.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.615+0000 7fd4f2d6a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4ec0cfe10 0x7fd4ec157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:00.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.615+0000 7fd4f2d6a640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd4ec15e790 0x7fd4ec0b8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:00.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.615+0000 7fd4f2d6a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4ec15eba0 0x7fd4ec0cc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:00.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.615+0000 7fd4f2d6a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec0b6710 con 0x7fd4ec0cfe10 2024-09-17T04:34:00.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.615+0000 7fd4f2d6a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec0b6590 con 0x7fd4ec15eba0 2024-09-17T04:34:00.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.615+0000 7fd4f2d6a640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fd4ec0b6890 con 0x7fd4ec15e790 2024-09-17T04:34:00.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.615+0000 7fd4f0b06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4ec0cfe10 0x7fd4ec157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:00.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.615+0000 7fd4f0b06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4ec0cfe10 0x7fd4ec157900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34900/0 (socket says 172.21.15.37:34900) 2024-09-17T04:34:00.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.615+0000 7fd4f0b06640 1 -- 172.21.15.37:0/2513600226 learned_addr learned my addr 172.21.15.37:0/2513600226 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:00.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.615+0000 7fd4f0b06640 1 -- 172.21.15.37:0/2513600226 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd4ec15e790 msgr2=0x7fd4ec0b8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:00.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.615+0000 7fd4f0b06640 1 --2- 172.21.15.37:0/2513600226 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd4ec15e790 0x7fd4ec0b8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.615+0000 7fd4f0b06640 1 -- 172.21.15.37:0/2513600226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4ec15eba0 msgr2=0x7fd4ec0cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:00.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.615+0000 7fd4f1307640 1 --2- 172.21.15.37:0/2513600226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4ec15eba0 0x7fd4ec0cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:00.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.615+0000 7fd4f0b06640 1 --2- 172.21.15.37:0/2513600226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4ec15eba0 0x7fd4ec0cc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.616+0000 7fd4f0b06640 1 -- 172.21.15.37:0/2513600226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ec1591c0 con 0x7fd4ec0cfe10 2024-09-17T04:34:00.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.616+0000 7fd4f0b06640 1 --2- 172.21.15.37:0/2513600226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4ec0cfe10 0x7fd4ec157900 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd4e000e7c0 tx=0x7fd4e000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:00.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.616+0000 7fd4e97fa640 1 -- 172.21.15.37:0/2513600226 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e000ccc0 con 0x7fd4ec0cfe10 2024-09-17T04:34:00.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.616+0000 7fd4e97fa640 1 -- 172.21.15.37:0/2513600226 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4e000ce80 con 0x7fd4ec0cfe10 2024-09-17T04:34:00.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.616+0000 7fd4f2d6a640 1 -- 172.21.15.37:0/2513600226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4ec1f58f0 con 0x7fd4ec0cfe10 2024-09-17T04:34:00.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.616+0000 7fd4e97fa640 1 -- 172.21.15.37:0/2513600226 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0005280 con 0x7fd4ec0cfe10 2024-09-17T04:34:00.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.616+0000 7fd4f2d6a640 1 -- 172.21.15.37:0/2513600226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4ec1f5eb0 con 0x7fd4ec0cfe10 2024-09-17T04:34:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.617+0000 7fd4f2d6a640 1 -- 172.21.15.37:0/2513600226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4ec159ac0 con 0x7fd4ec0cfe10 2024-09-17T04:34:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.618+0000 7fd4e97fa640 1 -- 172.21.15.37:0/2513600226 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd4e0005420 con 0x7fd4ec0cfe10 2024-09-17T04:34:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.618+0000 7fd4e97fa640 1 --2- 172.21.15.37:0/2513600226 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd4d0078b90 0x7fd4d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.619+0000 7fd4e97fa640 1 -- 172.21.15.37:0/2513600226 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fd4e009b700 con 0x7fd4ec0cfe10 2024-09-17T04:34:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.619+0000 7fd4ebfff640 1 --2- 172.21.15.37:0/2513600226 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd4d0078b90 0x7fd4d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.619+0000 7fd4ebfff640 1 --2- 172.21.15.37:0/2513600226 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd4d0078b90 0x7fd4d007b0a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8033000 tx=0x7fd4d80046c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:00.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.623+0000 7fd4e97fa640 1 -- 172.21.15.37:0/2513600226 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd4e0063b40 con 0x7fd4ec0cfe10 2024-09-17T04:34:00.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.807+0000 7fd4f2d6a640 1 -- 172.21.15.37:0/2513600226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4ec0b6d40 con 0x7fd4ec0cfe10 2024-09-17T04:34:00.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.808+0000 7fd4e97fa640 1 -- 172.21.15.37:0/2513600226 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16682 (secure 0 0 0) 0x7fd4e0063440 con 0x7fd4ec0cfe10 2024-09-17T04:34:00.808 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:00.808 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":32,"fsid":"a908a603-430c-4209-8294-0c3460b35f42","created":"2024-09-17T04:32:54.139221+0000","modified":"2024-09-17T04:33:59.490304+0000","last_up_change":"2024-09-17T04:33:06.922210+0000","last_in_change":"2024-09-17T04:32:57.754118+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:33:08.461907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:33:57.558905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:58.446650+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ea4b8d1-0d22-4dd9-b913-9df585ef24ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6809","nonce":2283587593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6811","nonce":2283587593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6815","nonce":2283587593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6813","nonce":2283587593}]},"public_addr":"172.21.15.37:6809/2283587593","cluster_addr":"172.21.15.37:6811/2283587593","heartbeat_back_addr":"172.21.15.37:6815/2283587593","heartbeat_front_addr":"172.21.15.37:6813/2283587593","state":["exists","up"]},{"osd":1,"uuid":"ac745a33-e8ad-4083-9c18-71bf81982b76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6825","nonce":2657955935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6827","nonce":2657955935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6831","nonce":2657955935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6829","nonce":2657955935}]},"public_addr":"172.21.15.37:6825/2657955935","cluster_addr":"172.21.15.37:6827/2657955935","heartbeat_back_addr":"172.21.15.37:6831/2657955935","heartbeat_front_addr":"172.21.15.37:6829/2657955935","state":["exists","up"]},{"osd":2,"uuid":"3c63f3d8-132a-4735-b0bc-d64313a46503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6817","nonce":579720176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6819","nonce":579720176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6823","nonce":579720176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6821","nonce":579720176}]},"public_addr":"172.21.15.37:6817/579720176","cluster_addr":"172.21.15.37:6819/579720176","heartbeat_back_addr":"172.21.15.37:6823/579720176","heartbeat_front_addr":"172.21.15.37:6821/579720176","state":["exists","up"]},{"osd":3,"uuid":"4f8c6ca2-ea1d-4610-9127-52c4dd8bc8e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6801","nonce":809975826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6803","nonce":809975826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6807","nonce":809975826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6805","nonce":809975826}]},"public_addr":"172.21.15.37:6801/809975826","cluster_addr":"172.21.15.37:6803/809975826","heartbeat_back_addr":"172.21.15.37:6807/809975826","heartbeat_front_addr":"172.21.15.37:6805/809975826","state":["exists","up"]},{"osd":4,"uuid":"63d0c4ab-f115-47b4-9611-12a10153473a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6819","nonce":2509785752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6821","nonce":2509785752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6825","nonce":2509785752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6823","nonce":2509785752}]},"public_addr":"172.21.15.92:6819/2509785752","cluster_addr":"172.21.15.92:6821/2509785752","heartbeat_back_addr":"172.21.15.92:6825/2509785752","heartbeat_front_addr":"172.21.15.92:6823/2509785752","state":["exists","up"]},{"osd":5,"uuid":"4c56c04d-0c07-4f76-9d75-6335121310ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6827","nonce":248263832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6829","nonce":248263832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6833","nonce":248263832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6831","nonce":248263832}]},"public_addr":"172.21.15.92:6827/248263832","cluster_addr":"172.21.15.92:6829/248263832","heartbeat_back_addr":"172.21.15.92:6833/248263832","heartbeat_front_addr":"172.21.15.92:6831/248263832","state":["exists","up"]},{"osd":6,"uuid":"b30eae7c-096a-4d24-ac51-96b8b13ecaee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6811","nonce":4078365985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6813","nonce":4078365985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6817","nonce":4078365985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6815","nonce":4078365985}]},"public_addr":"172.21.15.92:6811/4078365985","cluster_addr":"172.21.15.92:6813/4078365985","heartbeat_back_addr":"172.21.15.92:6817/4078365985","heartbeat_front_addr":"172.21.15.92:6815/4078365985","state":["exists","up"]},{"osd":7,"uuid":"62f12540-d374-4a4c-9ff7-63b7304fe77f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6803","nonce":3444787223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6805","nonce":3444787223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6809","nonce":3444787223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6807","nonce":3444787223}]},"public_addr":"172.21.15.92:6803/3444787223","cluster_addr":"172.21.15.92:6805/3444787223","heartbeat_back_addr":"172.21.15.92:6809/3444787223","heartbeat_front_addr":"172.21.15.92:6807/3444787223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.006420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.111135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.103325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.072656+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.200989+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.060455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.774773+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.865778+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:34:00.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.810+0000 7fd4f2d6a640 1 -- 172.21.15.37:0/2513600226 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd4d0078b90 msgr2=0x7fd4d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:00.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.810+0000 7fd4f2d6a640 1 --2- 172.21.15.37:0/2513600226 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd4d0078b90 0x7fd4d007b0a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8033000 tx=0x7fd4d80046c0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.810+0000 7fd4f2d6a640 1 -- 172.21.15.37:0/2513600226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4ec0cfe10 msgr2=0x7fd4ec157900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:00.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.810+0000 7fd4f2d6a640 1 --2- 172.21.15.37:0/2513600226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4ec0cfe10 0x7fd4ec157900 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd4e000e7c0 tx=0x7fd4e000ec80 comp rx=0 tx=0).stop 2024-09-17T04:34:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.810+0000 7fd4f2d6a640 1 -- 172.21.15.37:0/2513600226 shutdown_connections 2024-09-17T04:34:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.810+0000 7fd4f2d6a640 1 --2- 172.21.15.37:0/2513600226 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd4d0078b90 0x7fd4d007b0a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.810+0000 7fd4f2d6a640 1 --2- 172.21.15.37:0/2513600226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4ec15eba0 0x7fd4ec0cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.810+0000 7fd4f2d6a640 1 --2- 172.21.15.37:0/2513600226 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd4ec15e790 0x7fd4ec0b8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.810+0000 7fd4f2d6a640 1 --2- 172.21.15.37:0/2513600226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4ec0cfe10 0x7fd4ec157900 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.810+0000 7fd4f2d6a640 1 -- 172.21.15.37:0/2513600226 >> 172.21.15.37:0/2513600226 conn(0x7fd4ec06ddb0 msgr2=0x7fd4ec0b5680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.810+0000 7fd4f2d6a640 1 -- 172.21.15.37:0/2513600226 shutdown_connections 2024-09-17T04:34:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.810+0000 7fd4f2d6a640 1 -- 172.21.15.37:0/2513600226 wait complete. 2024-09-17T04:34:00.822 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T04:34:00.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.942+0000 7f81d608b640 1 -- 172.21.15.37:0/1829021032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81bc004800 msgr2=0x7f81bc004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:00.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.942+0000 7f81d608b640 1 --2- 172.21.15.37:0/1829021032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81bc004800 0x7f81bc004c10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f81b8009b50 tx=0x7f81bc005830 comp rx=0 tx=0).stop 2024-09-17T04:34:00.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.942+0000 7f81cffff640 1 -- 172.21.15.37:0/1829021032 reap_dead start 2024-09-17T04:34:00.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.942+0000 7f81d608b640 1 -- 172.21.15.37:0/1829021032 shutdown_connections 2024-09-17T04:34:00.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.942+0000 7f81d608b640 1 -- 172.21.15.37:0/1829021032 >> 172.21.15.37:0/1829021032 conn(0x7f81d00d44c0 msgr2=0x7f81d00d48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:00.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.942+0000 7f81d608b640 1 -- 172.21.15.37:0/1829021032 shutdown_connections 2024-09-17T04:34:00.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.942+0000 7f81d608b640 1 -- 172.21.15.37:0/1829021032 wait complete. 2024-09-17T04:34:00.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.944+0000 7f81d608b640 1 Processor -- start 2024-09-17T04:34:00.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.944+0000 7f81d608b640 1 -- start start 2024-09-17T04:34:00.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.946+0000 7f81d608b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81bc004800 0x7f81d01f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:00.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.946+0000 7f81d608b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f81d015ef10 0x7f81d0157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:00.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.946+0000 7f81d608b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81d015a020 0x7f81d015a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:00.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.946+0000 7f81d608b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f81d015e8e0 con 0x7f81bc004800 2024-09-17T04:34:00.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.946+0000 7f81d608b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f81d00b6860 con 0x7f81d015a020 2024-09-17T04:34:00.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.946+0000 7f81d608b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f81d015ea60 con 0x7f81d015ef10 2024-09-17T04:34:00.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.946+0000 7f81cf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81bc004800 0x7f81d01f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:00.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.946+0000 7f81cf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81bc004800 0x7f81d01f5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34916/0 (socket says 172.21.15.37:34916) 2024-09-17T04:34:00.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.946+0000 7f81cf7fe640 1 -- 172.21.15.37:0/4141751725 learned_addr learned my addr 172.21.15.37:0/4141751725 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:00.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.946+0000 7f81ceffd640 1 --2- 172.21.15.37:0/4141751725 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f81d015ef10 0x7f81d0157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:00.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.946+0000 7f81cffff640 1 --2- 172.21.15.37:0/4141751725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81d015a020 0x7f81d015a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:00.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.946+0000 7f81cf7fe640 1 -- 172.21.15.37:0/4141751725 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f81d015ef10 msgr2=0x7f81d0157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:00.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.946+0000 7f81cf7fe640 1 --2- 172.21.15.37:0/4141751725 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f81d015ef10 0x7f81d0157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.946+0000 7f81cf7fe640 1 -- 172.21.15.37:0/4141751725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81d015a020 msgr2=0x7f81d015a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:00.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.946+0000 7f81cf7fe640 1 --2- 172.21.15.37:0/4141751725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81d015a020 0x7f81d015a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:00.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.946+0000 7f81cf7fe640 1 -- 172.21.15.37:0/4141751725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d01618f0 con 0x7f81bc004800 2024-09-17T04:34:00.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.946+0000 7f81cf7fe640 1 --2- 172.21.15.37:0/4141751725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81bc004800 0x7f81d01f5b80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f81c0006110 tx=0x7f81c0006510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:00.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.947+0000 7f81c7fff640 1 -- 172.21.15.37:0/4141751725 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c000f8d0 con 0x7f81bc004800 2024-09-17T04:34:00.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.947+0000 7f81c7fff640 1 -- 172.21.15.37:0/4141751725 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81c0016020 con 0x7f81bc004800 2024-09-17T04:34:00.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.947+0000 7f81d608b640 1 -- 172.21.15.37:0/4141751725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81d0161bd0 con 0x7f81bc004800 2024-09-17T04:34:00.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.947+0000 7f81c7fff640 1 -- 172.21.15.37:0/4141751725 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c000fd10 con 0x7f81bc004800 2024-09-17T04:34:00.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.947+0000 7f81d608b640 1 -- 172.21.15.37:0/4141751725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81d015f3c0 con 0x7f81bc004800 2024-09-17T04:34:00.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.948+0000 7f81d608b640 1 -- 172.21.15.37:0/4141751725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81d00d3fe0 con 0x7f81bc004800 2024-09-17T04:34:00.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.949+0000 7f81c7fff640 1 -- 172.21.15.37:0/4141751725 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f81c0025440 con 0x7f81bc004800 2024-09-17T04:34:00.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.949+0000 7f81c7fff640 1 --2- 172.21.15.37:0/4141751725 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f81b0078b90 0x7f81b007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:00.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.949+0000 7f81c7fff640 1 -- 172.21.15.37:0/4141751725 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f81c009bde0 con 0x7f81bc004800 2024-09-17T04:34:00.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.949+0000 7f81ceffd640 1 --2- 172.21.15.37:0/4141751725 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f81b0078b90 0x7f81b007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:00.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.950+0000 7f81ceffd640 1 --2- 172.21.15.37:0/4141751725 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f81b0078b90 0x7f81b007b0a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f81b8035000 tx=0x7f81b80046c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:00.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:00.953+0000 7f81c7fff640 1 -- 172.21.15.37:0/4141751725 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f81c0020020 con 0x7f81bc004800 2024-09-17T04:34:01.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:01.179+0000 7f81d608b640 1 -- 172.21.15.37:0/4141751725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f81d00d4d90 con 0x7f81bc004800 2024-09-17T04:34:02.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.137+0000 7f81c7fff640 1 -- 172.21.15.37:0/4141751725 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f81c0064420 con 0x7f81bc004800 2024-09-17T04:34:02.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.141+0000 7f81d608b640 1 -- 172.21.15.37:0/4141751725 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f81b0078b90 msgr2=0x7f81b007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:02.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.141+0000 7f81d608b640 1 --2- 172.21.15.37:0/4141751725 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f81b0078b90 0x7f81b007b0a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f81b8035000 tx=0x7f81b80046c0 comp rx=0 tx=0).stop 2024-09-17T04:34:02.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.141+0000 7f81d608b640 1 -- 172.21.15.37:0/4141751725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81bc004800 msgr2=0x7f81d01f5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:02.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.141+0000 7f81d608b640 1 --2- 172.21.15.37:0/4141751725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81bc004800 0x7f81d01f5b80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f81c0006110 tx=0x7f81c0006510 comp rx=0 tx=0).stop 2024-09-17T04:34:02.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.141+0000 7f81d608b640 1 -- 172.21.15.37:0/4141751725 shutdown_connections 2024-09-17T04:34:02.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.141+0000 7f81d608b640 1 --2- 172.21.15.37:0/4141751725 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f81b0078b90 0x7f81b007b0a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:02.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.141+0000 7f81d608b640 1 --2- 172.21.15.37:0/4141751725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81d015a020 0x7f81d015a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:02.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.141+0000 7f81d608b640 1 --2- 172.21.15.37:0/4141751725 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f81d015ef10 0x7f81d0157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:02.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.141+0000 7f81d608b640 1 --2- 172.21.15.37:0/4141751725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81bc004800 0x7f81d01f5b80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:02.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.141+0000 7f81d608b640 1 -- 172.21.15.37:0/4141751725 >> 172.21.15.37:0/4141751725 conn(0x7f81d00d44c0 msgr2=0x7f81d00b9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:02.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.141+0000 7f81d608b640 1 -- 172.21.15.37:0/4141751725 shutdown_connections 2024-09-17T04:34:02.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.141+0000 7f81d608b640 1 -- 172.21.15.37:0/4141751725 wait complete. 2024-09-17T04:34:02.155 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T04:34:02.156 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T04:34:02.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.284+0000 7fb4bb41d640 1 -- 172.21.15.37:0/4250498040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb49c003ac0 msgr2=0x7fb49c003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:02.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.284+0000 7fb4bb41d640 1 --2- 172.21.15.37:0/4250498040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb49c003ac0 0x7fb49c003ef0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400c640 tx=0x7fb49c005830 comp rx=0 tx=0).stop 2024-09-17T04:34:02.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.284+0000 7fb4b99ba640 1 -- 172.21.15.37:0/4250498040 reap_dead start 2024-09-17T04:34:02.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.284+0000 7fb4bb41d640 1 -- 172.21.15.37:0/4250498040 shutdown_connections 2024-09-17T04:34:02.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.284+0000 7fb4bb41d640 1 -- 172.21.15.37:0/4250498040 >> 172.21.15.37:0/4250498040 conn(0x7fb4b40cede0 msgr2=0x7fb4b40cf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:02.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.284+0000 7fb4bb41d640 1 -- 172.21.15.37:0/4250498040 shutdown_connections 2024-09-17T04:34:02.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.284+0000 7fb4bb41d640 1 -- 172.21.15.37:0/4250498040 wait complete. 2024-09-17T04:34:02.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.286+0000 7fb4bb41d640 1 Processor -- start 2024-09-17T04:34:02.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.286+0000 7fb4bb41d640 1 -- start start 2024-09-17T04:34:02.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.287+0000 7fb4bb41d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb49c003ac0 0x7fb4b41f2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:02.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.287+0000 7fb4bb41d640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb4b40b78e0 0x7fb4b41f6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:02.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.287+0000 7fb4bb41d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4b41f2a50 0x7fb4b41f6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:02.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.287+0000 7fb4bb41d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b40cd450 con 0x7fb49c003ac0 2024-09-17T04:34:02.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.287+0000 7fb4bb41d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b40cd2d0 con 0x7fb4b41f2a50 2024-09-17T04:34:02.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.287+0000 7fb4bb41d640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fb4b415ceb0 con 0x7fb4b40b78e0 2024-09-17T04:34:02.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.287+0000 7fb4b91b9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb49c003ac0 0x7fb4b41f2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:02.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.287+0000 7fb4b91b9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb49c003ac0 0x7fb4b41f2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34938/0 (socket says 172.21.15.37:34938) 2024-09-17T04:34:02.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.287+0000 7fb4b91b9640 1 -- 172.21.15.37:0/4206307700 learned_addr learned my addr 172.21.15.37:0/4206307700 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:02.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.288+0000 7fb4b99ba640 1 --2- 172.21.15.37:0/4206307700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4b41f2a50 0x7fb4b41f6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:02.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.288+0000 7fb4b89b8640 1 --2- 172.21.15.37:0/4206307700 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb4b40b78e0 0x7fb4b41f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:02.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.288+0000 7fb4b91b9640 1 -- 172.21.15.37:0/4206307700 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb4b40b78e0 msgr2=0x7fb4b41f6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:02.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.288+0000 7fb4b91b9640 1 --2- 172.21.15.37:0/4206307700 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb4b40b78e0 0x7fb4b41f6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:02.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.288+0000 7fb4b91b9640 1 -- 172.21.15.37:0/4206307700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4b41f2a50 msgr2=0x7fb4b41f6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:02.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.288+0000 7fb4b91b9640 1 --2- 172.21.15.37:0/4206307700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4b41f2a50 0x7fb4b41f6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:02.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.288+0000 7fb4b91b9640 1 -- 172.21.15.37:0/4206307700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b41f71b0 con 0x7fb49c003ac0 2024-09-17T04:34:02.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.288+0000 7fb4b89b8640 1 --2- 172.21.15.37:0/4206307700 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb4b40b78e0 0x7fb4b41f6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:02.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.288+0000 7fb4b91b9640 1 --2- 172.21.15.37:0/4206307700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb49c003ac0 0x7fb4b41f2510 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb4a402b490 tx=0x7fb4a400c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:02.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.288+0000 7fb4b1ffb640 1 -- 172.21.15.37:0/4206307700 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4039070 con 0x7fb49c003ac0 2024-09-17T04:34:02.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.288+0000 7fb4b1ffb640 1 -- 172.21.15.37:0/4206307700 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4a4003b80 con 0x7fb49c003ac0 2024-09-17T04:34:02.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.288+0000 7fb4b1ffb640 1 -- 172.21.15.37:0/4206307700 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a402b7e0 con 0x7fb49c003ac0 2024-09-17T04:34:02.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.288+0000 7fb4bb41d640 1 -- 172.21.15.37:0/4206307700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b40b6e00 con 0x7fb49c003ac0 2024-09-17T04:34:02.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.288+0000 7fb4bb41d640 1 -- 172.21.15.37:0/4206307700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b40b73c0 con 0x7fb49c003ac0 2024-09-17T04:34:02.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.289+0000 7fb4bb41d640 1 -- 172.21.15.37:0/4206307700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4b40b6bf0 con 0x7fb49c003ac0 2024-09-17T04:34:02.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.290+0000 7fb4b1ffb640 1 -- 172.21.15.37:0/4206307700 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb4a402b980 con 0x7fb49c003ac0 2024-09-17T04:34:02.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.290+0000 7fb4b1ffb640 1 --2- 172.21.15.37:0/4206307700 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb4a0078b90 0x7fb4a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:02.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.291+0000 7fb4b1ffb640 1 -- 172.21.15.37:0/4206307700 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fb4a40ba8a0 con 0x7fb49c003ac0 2024-09-17T04:34:02.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.291+0000 7fb4b89b8640 1 --2- 172.21.15.37:0/4206307700 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb4a0078b90 0x7fb4a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:02.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.291+0000 7fb4b89b8640 1 --2- 172.21.15.37:0/4206307700 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb4a0078b90 0x7fb4a007b0a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb4a80069b0 tx=0x7fb4a8006a60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:02.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.295+0000 7fb4b1ffb640 1 -- 172.21.15.37:0/4206307700 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb4a4046030 con 0x7fb49c003ac0 2024-09-17T04:34:02.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:02.517+0000 7fb4bb41d640 1 -- 172.21.15.37:0/4206307700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb4b40ccbc0 con 0x7fb49c003ac0 2024-09-17T04:34:03.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.143+0000 7fb4b1ffb640 1 -- 172.21.15.37:0/4206307700 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7fb4a4082c40 con 0x7fb49c003ac0 2024-09-17T04:34:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.146+0000 7fb4bb41d640 1 -- 172.21.15.37:0/4206307700 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb4a0078b90 msgr2=0x7fb4a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.146+0000 7fb4bb41d640 1 --2- 172.21.15.37:0/4206307700 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb4a0078b90 0x7fb4a007b0a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb4a80069b0 tx=0x7fb4a8006a60 comp rx=0 tx=0).stop 2024-09-17T04:34:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.146+0000 7fb4bb41d640 1 -- 172.21.15.37:0/4206307700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb49c003ac0 msgr2=0x7fb4b41f2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.146+0000 7fb4bb41d640 1 --2- 172.21.15.37:0/4206307700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb49c003ac0 0x7fb4b41f2510 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb4a402b490 tx=0x7fb4a400c260 comp rx=0 tx=0).stop 2024-09-17T04:34:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.146+0000 7fb4bb41d640 1 -- 172.21.15.37:0/4206307700 shutdown_connections 2024-09-17T04:34:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.146+0000 7fb4bb41d640 1 --2- 172.21.15.37:0/4206307700 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb4a0078b90 0x7fb4a007b0a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.146+0000 7fb4bb41d640 1 --2- 172.21.15.37:0/4206307700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4b41f2a50 0x7fb4b41f6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.146+0000 7fb4bb41d640 1 --2- 172.21.15.37:0/4206307700 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb4b40b78e0 0x7fb4b41f6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.146+0000 7fb4bb41d640 1 --2- 172.21.15.37:0/4206307700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb49c003ac0 0x7fb4b41f2510 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.146+0000 7fb4bb41d640 1 -- 172.21.15.37:0/4206307700 >> 172.21.15.37:0/4206307700 conn(0x7fb4b40cede0 msgr2=0x7fb4b4075b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.146+0000 7fb4bb41d640 1 -- 172.21.15.37:0/4206307700 shutdown_connections 2024-09-17T04:34:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.146+0000 7fb4bb41d640 1 -- 172.21.15.37:0/4206307700 wait complete. 2024-09-17T04:34:03.161 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T04:34:03.161 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:34:03.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.285+0000 7f3022594640 1 -- 172.21.15.37:0/2060356009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f301c0d0cc0 msgr2=0x7f301c0cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:03.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.285+0000 7f3022594640 1 --2- 172.21.15.37:0/2060356009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f301c0d0cc0 0x7f301c0cd880 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3010009830 tx=0x7f301002a5a0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.285+0000 7f3022594640 1 -- 172.21.15.37:0/2060356009 shutdown_connections 2024-09-17T04:34:03.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.285+0000 7f3022594640 1 --2- 172.21.15.37:0/2060356009 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f301c0cddc0 0x7f301c0aea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.285+0000 7f3022594640 1 --2- 172.21.15.37:0/2060356009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f301c0d0cc0 0x7f301c0cd880 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.285+0000 7f3022594640 1 -- 172.21.15.37:0/2060356009 >> 172.21.15.37:0/2060356009 conn(0x7f301c0b65d0 msgr2=0x7f301c0b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:03.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.285+0000 7f3022594640 1 -- 172.21.15.37:0/2060356009 shutdown_connections 2024-09-17T04:34:03.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.285+0000 7f3022594640 1 -- 172.21.15.37:0/2060356009 wait complete. 2024-09-17T04:34:03.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.287+0000 7f3022594640 1 Processor -- start 2024-09-17T04:34:03.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.287+0000 7f3022594640 1 -- start start 2024-09-17T04:34:03.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.289+0000 7f3022594640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f301c0b69e0 0x7f301c146ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:03.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.289+0000 7f3022594640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f301c0cddc0 0x7f301c156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:03.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.289+0000 7f3022594640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f301c0d0cc0 0x7f301c1573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:03.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.289+0000 7f3022594640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f301c0af300 con 0x7f301c0cddc0 2024-09-17T04:34:03.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.289+0000 7f3022594640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f301c0af180 con 0x7f301c0d0cc0 2024-09-17T04:34:03.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.289+0000 7f3022594640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f301c0af480 con 0x7f301c0b69e0 2024-09-17T04:34:03.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.289+0000 7f301b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f301c0cddc0 0x7f301c156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:03.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.289+0000 7f301b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f301c0cddc0 0x7f301c156e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34952/0 (socket says 172.21.15.37:34952) 2024-09-17T04:34:03.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.289+0000 7f301b7fe640 1 -- 172.21.15.37:0/3137182763 learned_addr learned my addr 172.21.15.37:0/3137182763 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:03.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.289+0000 7f301bfff640 1 --2- 172.21.15.37:0/3137182763 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f301c0b69e0 0x7f301c146ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:03.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.289+0000 7f3020b31640 1 --2- 172.21.15.37:0/3137182763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f301c0d0cc0 0x7f301c1573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:03.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.289+0000 7f301b7fe640 1 -- 172.21.15.37:0/3137182763 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f301c0b69e0 msgr2=0x7f301c146ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:03.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.289+0000 7f301b7fe640 1 --2- 172.21.15.37:0/3137182763 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f301c0b69e0 0x7f301c146ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.289+0000 7f301b7fe640 1 -- 172.21.15.37:0/3137182763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f301c0d0cc0 msgr2=0x7f301c1573d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:03.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.289+0000 7f301b7fe640 1 --2- 172.21.15.37:0/3137182763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f301c0d0cc0 0x7f301c1573d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.289+0000 7f301b7fe640 1 -- 172.21.15.37:0/3137182763 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f301c1f94f0 con 0x7f301c0cddc0 2024-09-17T04:34:03.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.289+0000 7f301bfff640 1 --2- 172.21.15.37:0/3137182763 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f301c0b69e0 0x7f301c146ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:03.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.289+0000 7f301b7fe640 1 --2- 172.21.15.37:0/3137182763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f301c0cddc0 0x7f301c156e90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3010009950 tx=0x7f3010033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:03.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.290+0000 7f3018ff9640 1 -- 172.21.15.37:0/3137182763 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010031d60 con 0x7f301c0cddc0 2024-09-17T04:34:03.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.290+0000 7f3022594640 1 -- 172.21.15.37:0/3137182763 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f301c1f97d0 con 0x7f301c0cddc0 2024-09-17T04:34:03.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.290+0000 7f3018ff9640 1 -- 172.21.15.37:0/3137182763 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3010043070 con 0x7f301c0cddc0 2024-09-17T04:34:03.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.290+0000 7f3018ff9640 1 -- 172.21.15.37:0/3137182763 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010033290 con 0x7f301c0cddc0 2024-09-17T04:34:03.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.290+0000 7f3022594640 1 -- 172.21.15.37:0/3137182763 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f301c1f8f80 con 0x7f301c0cddc0 2024-09-17T04:34:03.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.291+0000 7f3022594640 1 -- 172.21.15.37:0/3137182763 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f301c0afa10 con 0x7f301c0cddc0 2024-09-17T04:34:03.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.292+0000 7f3018ff9640 1 -- 172.21.15.37:0/3137182763 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3010033430 con 0x7f301c0cddc0 2024-09-17T04:34:03.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.292+0000 7f3018ff9640 1 --2- 172.21.15.37:0/3137182763 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3008078b90 0x7f300807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:03.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.293+0000 7f3018ff9640 1 -- 172.21.15.37:0/3137182763 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f30100bbd40 con 0x7f301c0cddc0 2024-09-17T04:34:03.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.293+0000 7f301bfff640 1 --2- 172.21.15.37:0/3137182763 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3008078b90 0x7f300807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:03.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.293+0000 7f301bfff640 1 --2- 172.21.15.37:0/3137182763 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3008078b90 0x7f300807b0a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f300c0016a0 tx=0x7f300c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:03.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.297+0000 7f3018ff9640 1 -- 172.21.15.37:0/3137182763 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3010084100 con 0x7f301c0cddc0 2024-09-17T04:34:03.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.522+0000 7f3022594640 1 -- 172.21.15.37:0/3137182763 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f301c1586c0 con 0x7f301c0cddc0 2024-09-17T04:34:03.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.522+0000 7f3018ff9640 1 -- 172.21.15.37:0/3137182763 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5223 (secure 0 0 0) 0x7f3010083a00 con 0x7f301c0cddc0 2024-09-17T04:34:03.523 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:03.523 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"btime":"2024-09-17T04:34:03:138395+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4870,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/4152799414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4152799414},{"type":"v1","addr":"172.21.15.92:6836","nonce":4152799414}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4873,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/3730800721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":3730800721},{"type":"v1","addr":"172.21.15.92:6837","nonce":3730800721}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/1682438485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1682438485},{"type":"v1","addr":"172.21.15.37:6834","nonce":1682438485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:59.490265+0000","modified":"2024-09-17T04:34:02.518988+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1358600605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1358600605},{"type":"v1","addr":"172.21.15.37:6835","nonce":1358600605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4953,"qdb_cluster":[4953]},"id":2}]} 2024-09-17T04:34:03.523 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 17 2024-09-17T04:34:03.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.525+0000 7f3022594640 1 -- 172.21.15.37:0/3137182763 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3008078b90 msgr2=0x7f300807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:03.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.525+0000 7f3022594640 1 --2- 172.21.15.37:0/3137182763 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3008078b90 0x7f300807b0a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f300c0016a0 tx=0x7f300c009040 comp rx=0 tx=0).stop 2024-09-17T04:34:03.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.525+0000 7f3022594640 1 -- 172.21.15.37:0/3137182763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f301c0cddc0 msgr2=0x7f301c156e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:03.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.525+0000 7f3022594640 1 --2- 172.21.15.37:0/3137182763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f301c0cddc0 0x7f301c156e90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3010009950 tx=0x7f3010033c20 comp rx=0 tx=0).stop 2024-09-17T04:34:03.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.525+0000 7f3022594640 1 -- 172.21.15.37:0/3137182763 shutdown_connections 2024-09-17T04:34:03.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.525+0000 7f3022594640 1 --2- 172.21.15.37:0/3137182763 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3008078b90 0x7f300807b0a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.525+0000 7f3022594640 1 --2- 172.21.15.37:0/3137182763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f301c0d0cc0 0x7f301c1573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.525+0000 7f3022594640 1 --2- 172.21.15.37:0/3137182763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f301c0cddc0 0x7f301c156e90 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.525+0000 7f3022594640 1 --2- 172.21.15.37:0/3137182763 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f301c0b69e0 0x7f301c146ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.525+0000 7f3022594640 1 -- 172.21.15.37:0/3137182763 >> 172.21.15.37:0/3137182763 conn(0x7f301c0b65d0 msgr2=0x7f301c0b4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:03.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.525+0000 7f3022594640 1 -- 172.21.15.37:0/3137182763 shutdown_connections 2024-09-17T04:34:03.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.525+0000 7f3022594640 1 -- 172.21.15.37:0/3137182763 wait complete. 2024-09-17T04:34:03.537 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:34:03.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.653+0000 7f9e05a9b640 1 -- 172.21.15.37:0/3389771799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e0015c630 msgr2=0x7f9e000cff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:03.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.654+0000 7f9e05a9b640 1 --2- 172.21.15.37:0/3389771799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e0015c630 0x7f9e000cff30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9df00098e0 tx=0x7f9df002a530 comp rx=0 tx=0).stop 2024-09-17T04:34:03.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.654+0000 7f9e05a9b640 1 -- 172.21.15.37:0/3389771799 shutdown_connections 2024-09-17T04:34:03.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.654+0000 7f9e05a9b640 1 --2- 172.21.15.37:0/3389771799 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9e000d19c0 0x7f9e000d1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.654+0000 7f9e05a9b640 1 --2- 172.21.15.37:0/3389771799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e0015c630 0x7f9e000cff30 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.654+0000 7f9e05a9b640 1 -- 172.21.15.37:0/3389771799 >> 172.21.15.37:0/3389771799 conn(0x7f9e000cede0 msgr2=0x7f9e000cf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:03.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.654+0000 7f9e05a9b640 1 -- 172.21.15.37:0/3389771799 shutdown_connections 2024-09-17T04:34:03.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.654+0000 7f9e05a9b640 1 -- 172.21.15.37:0/3389771799 wait complete. 2024-09-17T04:34:03.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.655+0000 7f9e05a9b640 1 Processor -- start 2024-09-17T04:34:03.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.656+0000 7f9e05a9b640 1 -- start start 2024-09-17T04:34:03.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.657+0000 7f9e05a9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e000ccbc0 0x7f9e00149a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:03.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.657+0000 7f9e05a9b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9e000d19c0 0x7f9e000cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:03.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.657+0000 7f9e05a9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e0015c630 0x7f9e0015f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:03.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.657+0000 7f9e05a9b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0015d2c0 con 0x7f9e0015c630 2024-09-17T04:34:03.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.657+0000 7f9e05a9b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0015d140 con 0x7f9e000ccbc0 2024-09-17T04:34:03.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.657+0000 7f9e05a9b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f9e0015d440 con 0x7f9e000d19c0 2024-09-17T04:34:03.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.657+0000 7f9dfffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e0015c630 0x7f9e0015f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:03.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.657+0000 7f9dfffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e0015c630 0x7f9e0015f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34980/0 (socket says 172.21.15.37:34980) 2024-09-17T04:34:03.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.657+0000 7f9dfffff640 1 -- 172.21.15.37:0/2805021319 learned_addr learned my addr 172.21.15.37:0/2805021319 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:03.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.657+0000 7f9dfeffd640 1 --2- 172.21.15.37:0/2805021319 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9e000d19c0 0x7f9e000cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:03.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.657+0000 7f9dff7fe640 1 --2- 172.21.15.37:0/2805021319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e000ccbc0 0x7f9e00149a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:03.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.657+0000 7f9dfffff640 1 -- 172.21.15.37:0/2805021319 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9e000d19c0 msgr2=0x7f9e000cbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:03.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.657+0000 7f9dfffff640 1 --2- 172.21.15.37:0/2805021319 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9e000d19c0 0x7f9e000cbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.657+0000 7f9dfffff640 1 -- 172.21.15.37:0/2805021319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e000ccbc0 msgr2=0x7f9e00149a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:03.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.657+0000 7f9dfffff640 1 --2- 172.21.15.37:0/2805021319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e000ccbc0 0x7f9e00149a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.657+0000 7f9dfffff640 1 -- 172.21.15.37:0/2805021319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e00150cc0 con 0x7f9e0015c630 2024-09-17T04:34:03.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.657+0000 7f9dfeffd640 1 --2- 172.21.15.37:0/2805021319 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9e000d19c0 0x7f9e000cbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:03.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.658+0000 7f9dfffff640 1 --2- 172.21.15.37:0/2805021319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e0015c630 0x7f9e0015f110 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9df800c730 tx=0x7f9df800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:03.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.658+0000 7f9de7fff640 1 -- 172.21.15.37:0/2805021319 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df8017070 con 0x7f9e0015c630 2024-09-17T04:34:03.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.658+0000 7f9e05a9b640 1 -- 172.21.15.37:0/2805021319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e0006e300 con 0x7f9e0015c630 2024-09-17T04:34:03.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.658+0000 7f9de7fff640 1 -- 172.21.15.37:0/2805021319 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9df8003c60 con 0x7f9e0015c630 2024-09-17T04:34:03.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.658+0000 7f9de7fff640 1 -- 172.21.15.37:0/2805021319 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df80052d0 con 0x7f9e0015c630 2024-09-17T04:34:03.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.658+0000 7f9e05a9b640 1 -- 172.21.15.37:0/2805021319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e0006ebb0 con 0x7f9e0015c630 2024-09-17T04:34:03.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.659+0000 7f9e05a9b640 1 -- 172.21.15.37:0/2805021319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e0015d910 con 0x7f9e0015c630 2024-09-17T04:34:03.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.659+0000 7f9de7fff640 1 -- 172.21.15.37:0/2805021319 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9df801a070 con 0x7f9e0015c630 2024-09-17T04:34:03.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.660+0000 7f9de7fff640 1 --2- 172.21.15.37:0/2805021319 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9dec078b90 0x7f9dec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:03.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.660+0000 7f9de7fff640 1 -- 172.21.15.37:0/2805021319 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f9df809a3c0 con 0x7f9e0015c630 2024-09-17T04:34:03.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.660+0000 7f9dff7fe640 1 --2- 172.21.15.37:0/2805021319 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9dec078b90 0x7f9dec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:03.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.661+0000 7f9dff7fe640 1 --2- 172.21.15.37:0/2805021319 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9dec078b90 0x7f9dec07b0a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9df0034000 tx=0x7f9df0004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:03.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.665+0000 7f9de7fff640 1 -- 172.21.15.37:0/2805021319 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9df8062810 con 0x7f9e0015c630 2024-09-17T04:34:03.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.850+0000 7f9e05a9b640 1 -- 172.21.15.37:0/2805021319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e000cebc0 con 0x7f9e0015c630 2024-09-17T04:34:03.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.852+0000 7f9de7fff640 1 -- 172.21.15.37:0/2805021319 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16669 (secure 0 0 0) 0x7f9df8062110 con 0x7f9e0015c630 2024-09-17T04:34:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:03.852 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":33,"fsid":"a908a603-430c-4209-8294-0c3460b35f42","created":"2024-09-17T04:32:54.139221+0000","modified":"2024-09-17T04:34:01.126280+0000","last_up_change":"2024-09-17T04:33:06.922210+0000","last_in_change":"2024-09-17T04:32:57.754118+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:33:08.461907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:33:57.558905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:58.446650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ea4b8d1-0d22-4dd9-b913-9df585ef24ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6809","nonce":2283587593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6811","nonce":2283587593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6815","nonce":2283587593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6813","nonce":2283587593}]},"public_addr":"172.21.15.37:6809/2283587593","cluster_addr":"172.21.15.37:6811/2283587593","heartbeat_back_addr":"172.21.15.37:6815/2283587593","heartbeat_front_addr":"172.21.15.37:6813/2283587593","state":["exists","up"]},{"osd":1,"uuid":"ac745a33-e8ad-4083-9c18-71bf81982b76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6825","nonce":2657955935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6827","nonce":2657955935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6831","nonce":2657955935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6829","nonce":2657955935}]},"public_addr":"172.21.15.37:6825/2657955935","cluster_addr":"172.21.15.37:6827/2657955935","heartbeat_back_addr":"172.21.15.37:6831/2657955935","heartbeat_front_addr":"172.21.15.37:6829/2657955935","state":["exists","up"]},{"osd":2,"uuid":"3c63f3d8-132a-4735-b0bc-d64313a46503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6817","nonce":579720176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6819","nonce":579720176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6823","nonce":579720176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6821","nonce":579720176}]},"public_addr":"172.21.15.37:6817/579720176","cluster_addr":"172.21.15.37:6819/579720176","heartbeat_back_addr":"172.21.15.37:6823/579720176","heartbeat_front_addr":"172.21.15.37:6821/579720176","state":["exists","up"]},{"osd":3,"uuid":"4f8c6ca2-ea1d-4610-9127-52c4dd8bc8e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6801","nonce":809975826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6803","nonce":809975826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6807","nonce":809975826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6805","nonce":809975826}]},"public_addr":"172.21.15.37:6801/809975826","cluster_addr":"172.21.15.37:6803/809975826","heartbeat_back_addr":"172.21.15.37:6807/809975826","heartbeat_front_addr":"172.21.15.37:6805/809975826","state":["exists","up"]},{"osd":4,"uuid":"63d0c4ab-f115-47b4-9611-12a10153473a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6819","nonce":2509785752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6821","nonce":2509785752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6825","nonce":2509785752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6823","nonce":2509785752}]},"public_addr":"172.21.15.92:6819/2509785752","cluster_addr":"172.21.15.92:6821/2509785752","heartbeat_back_addr":"172.21.15.92:6825/2509785752","heartbeat_front_addr":"172.21.15.92:6823/2509785752","state":["exists","up"]},{"osd":5,"uuid":"4c56c04d-0c07-4f76-9d75-6335121310ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6827","nonce":248263832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6829","nonce":248263832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6833","nonce":248263832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6831","nonce":248263832}]},"public_addr":"172.21.15.92:6827/248263832","cluster_addr":"172.21.15.92:6829/248263832","heartbeat_back_addr":"172.21.15.92:6833/248263832","heartbeat_front_addr":"172.21.15.92:6831/248263832","state":["exists","up"]},{"osd":6,"uuid":"b30eae7c-096a-4d24-ac51-96b8b13ecaee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6811","nonce":4078365985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6813","nonce":4078365985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6817","nonce":4078365985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6815","nonce":4078365985}]},"public_addr":"172.21.15.92:6811/4078365985","cluster_addr":"172.21.15.92:6813/4078365985","heartbeat_back_addr":"172.21.15.92:6817/4078365985","heartbeat_front_addr":"172.21.15.92:6815/4078365985","state":["exists","up"]},{"osd":7,"uuid":"62f12540-d374-4a4c-9ff7-63b7304fe77f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6803","nonce":3444787223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6805","nonce":3444787223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6809","nonce":3444787223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6807","nonce":3444787223}]},"public_addr":"172.21.15.92:6803/3444787223","cluster_addr":"172.21.15.92:6805/3444787223","heartbeat_back_addr":"172.21.15.92:6809/3444787223","heartbeat_front_addr":"172.21.15.92:6807/3444787223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.006420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.111135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.103325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.072656+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.200989+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.060455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.774773+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.865778+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:34:03.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.854+0000 7f9e05a9b640 1 -- 172.21.15.37:0/2805021319 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9dec078b90 msgr2=0x7f9dec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:03.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.854+0000 7f9e05a9b640 1 --2- 172.21.15.37:0/2805021319 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9dec078b90 0x7f9dec07b0a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9df0034000 tx=0x7f9df0004730 comp rx=0 tx=0).stop 2024-09-17T04:34:03.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.854+0000 7f9e05a9b640 1 -- 172.21.15.37:0/2805021319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e0015c630 msgr2=0x7f9e0015f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:03.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.854+0000 7f9e05a9b640 1 --2- 172.21.15.37:0/2805021319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e0015c630 0x7f9e0015f110 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9df800c730 tx=0x7f9df800cbf0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.854+0000 7f9e05a9b640 1 -- 172.21.15.37:0/2805021319 shutdown_connections 2024-09-17T04:34:03.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.854+0000 7f9e05a9b640 1 --2- 172.21.15.37:0/2805021319 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9dec078b90 0x7f9dec07b0a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.854+0000 7f9e05a9b640 1 --2- 172.21.15.37:0/2805021319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e0015c630 0x7f9e0015f110 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.854+0000 7f9e05a9b640 1 --2- 172.21.15.37:0/2805021319 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9e000d19c0 0x7f9e000cbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.854+0000 7f9e05a9b640 1 --2- 172.21.15.37:0/2805021319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e000ccbc0 0x7f9e00149a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.854+0000 7f9e05a9b640 1 -- 172.21.15.37:0/2805021319 >> 172.21.15.37:0/2805021319 conn(0x7f9e000cede0 msgr2=0x7f9e0015b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:03.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.854+0000 7f9e05a9b640 1 -- 172.21.15.37:0/2805021319 shutdown_connections 2024-09-17T04:34:03.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.854+0000 7f9e05a9b640 1 -- 172.21.15.37:0/2805021319 wait complete. 2024-09-17T04:34:03.865 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T04:34:03.866 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:34:03.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.982+0000 7f691dd2b640 1 -- 172.21.15.37:0/1019919173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6918163040 msgr2=0x7f69180cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:03.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.982+0000 7f691dd2b640 1 --2- 172.21.15.37:0/1019919173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6918163040 0x7f69180cf7e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f690c009830 tx=0x7f690c02a510 comp rx=0 tx=0).stop 2024-09-17T04:34:03.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.982+0000 7f691dd2b640 1 -- 172.21.15.37:0/1019919173 shutdown_connections 2024-09-17T04:34:03.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.982+0000 7f691dd2b640 1 --2- 172.21.15.37:0/1019919173 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f69180d0ff0 0x7f69180d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.982+0000 7f691dd2b640 1 --2- 172.21.15.37:0/1019919173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6918163040 0x7f69180cf7e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.982+0000 7f691dd2b640 1 -- 172.21.15.37:0/1019919173 >> 172.21.15.37:0/1019919173 conn(0x7f691806de00 msgr2=0x7f691806e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:03.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.982+0000 7f691dd2b640 1 -- 172.21.15.37:0/1019919173 shutdown_connections 2024-09-17T04:34:03.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.982+0000 7f691dd2b640 1 -- 172.21.15.37:0/1019919173 wait complete. 2024-09-17T04:34:03.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.984+0000 7f691dd2b640 1 Processor -- start 2024-09-17T04:34:03.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.984+0000 7f691dd2b640 1 -- start start 2024-09-17T04:34:03.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.986+0000 7f691dd2b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69180d0ff0 0x7f69180ce470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:03.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.986+0000 7f691dd2b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f691815fbe0 0x7f69180b6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:03.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.986+0000 7f691dd2b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6918163040 0x7f691814d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:03.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.986+0000 7f691dd2b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f69180d32a0 con 0x7f69180d0ff0 2024-09-17T04:34:03.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.986+0000 7f691dd2b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f69180d3120 con 0x7f691815fbe0 2024-09-17T04:34:03.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.986+0000 7f691dd2b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f69180d3420 con 0x7f6918163040 2024-09-17T04:34:03.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.986+0000 7f69177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69180d0ff0 0x7f69180ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:03.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.986+0000 7f69177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69180d0ff0 0x7f69180ce470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34982/0 (socket says 172.21.15.37:34982) 2024-09-17T04:34:03.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.986+0000 7f69177fe640 1 -- 172.21.15.37:0/1753090106 learned_addr learned my addr 172.21.15.37:0/1753090106 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:03.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.986+0000 7f6916ffd640 1 --2- 172.21.15.37:0/1753090106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f691815fbe0 0x7f69180b6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:03.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.986+0000 7f6917fff640 1 --2- 172.21.15.37:0/1753090106 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6918163040 0x7f691814d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:03.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.986+0000 7f69177fe640 1 -- 172.21.15.37:0/1753090106 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6918163040 msgr2=0x7f691814d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:03.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.986+0000 7f69177fe640 1 --2- 172.21.15.37:0/1753090106 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6918163040 0x7f691814d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.986+0000 7f69177fe640 1 -- 172.21.15.37:0/1753090106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f691815fbe0 msgr2=0x7f69180b6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:03.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.986+0000 7f69177fe640 1 --2- 172.21.15.37:0/1753090106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f691815fbe0 0x7f69180b6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:03.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.986+0000 7f69177fe640 1 -- 172.21.15.37:0/1753090106 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69180cd3d0 con 0x7f69180d0ff0 2024-09-17T04:34:03.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.986+0000 7f69177fe640 1 --2- 172.21.15.37:0/1753090106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69180d0ff0 0x7f69180ce470 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f690800c730 tx=0x7f690800cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:03.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.987+0000 7f68fffff640 1 -- 172.21.15.37:0/1753090106 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6908016020 con 0x7f69180d0ff0 2024-09-17T04:34:03.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.987+0000 7f68fffff640 1 -- 172.21.15.37:0/1753090106 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f690800a950 con 0x7f69180d0ff0 2024-09-17T04:34:03.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.987+0000 7f691dd2b640 1 -- 172.21.15.37:0/1753090106 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69180cd6b0 con 0x7f69180d0ff0 2024-09-17T04:34:03.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.987+0000 7f68fffff640 1 -- 172.21.15.37:0/1753090106 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690800ada0 con 0x7f69180d0ff0 2024-09-17T04:34:03.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.987+0000 7f691dd2b640 1 -- 172.21.15.37:0/1753090106 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69180cdc70 con 0x7f69180d0ff0 2024-09-17T04:34:03.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.988+0000 7f691dd2b640 1 -- 172.21.15.37:0/1753090106 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69180d3940 con 0x7f69180d0ff0 2024-09-17T04:34:03.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.989+0000 7f68fffff640 1 -- 172.21.15.37:0/1753090106 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6908019020 con 0x7f69180d0ff0 2024-09-17T04:34:03.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.989+0000 7f68fffff640 1 --2- 172.21.15.37:0/1753090106 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6904078b90 0x7f690407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:03.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.989+0000 7f68fffff640 1 -- 172.21.15.37:0/1753090106 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f690809a870 con 0x7f69180d0ff0 2024-09-17T04:34:03.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.989+0000 7f6916ffd640 1 --2- 172.21.15.37:0/1753090106 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6904078b90 0x7f690407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:03.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.990+0000 7f6916ffd640 1 --2- 172.21.15.37:0/1753090106 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6904078b90 0x7f690407b0a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f690c034000 tx=0x7f690c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:03.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:03.994+0000 7f68fffff640 1 -- 172.21.15.37:0/1753090106 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6908020030 con 0x7f69180d0ff0 2024-09-17T04:34:04.029 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-09-17T04:34:04.029+0000 7f8bf8a92640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T04:34:04.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.176+0000 7f691dd2b640 1 -- 172.21.15.37:0/1753090106 --> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f691806a1a0 con 0x7f6904078b90 2024-09-17T04:34:04.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.181+0000 7f68fffff640 1 -- 172.21.15.37:0/1753090106 <== mgr.4100 v2:172.21.15.92:6800/864337835 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251939 (secure 0 0 0) 0x7f690c009450 con 0x7f6904078b90 2024-09-17T04:34:04.184 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:04.186 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-17T04:34:04.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.184+0000 7f691dd2b640 1 -- 172.21.15.37:0/1753090106 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6904078b90 msgr2=0x7f690407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:04.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.184+0000 7f691dd2b640 1 --2- 172.21.15.37:0/1753090106 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6904078b90 0x7f690407b0a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f690c034000 tx=0x7f690c004730 comp rx=0 tx=0).stop 2024-09-17T04:34:04.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.184+0000 7f691dd2b640 1 -- 172.21.15.37:0/1753090106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69180d0ff0 msgr2=0x7f69180ce470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:04.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.185+0000 7f691dd2b640 1 --2- 172.21.15.37:0/1753090106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69180d0ff0 0x7f69180ce470 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f690800c730 tx=0x7f690800cb30 comp rx=0 tx=0).stop 2024-09-17T04:34:04.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.185+0000 7f691dd2b640 1 -- 172.21.15.37:0/1753090106 shutdown_connections 2024-09-17T04:34:04.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.185+0000 7f691dd2b640 1 --2- 172.21.15.37:0/1753090106 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6904078b90 0x7f690407b0a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.185+0000 7f691dd2b640 1 --2- 172.21.15.37:0/1753090106 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6918163040 0x7f691814d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.185+0000 7f691dd2b640 1 --2- 172.21.15.37:0/1753090106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f691815fbe0 0x7f69180b6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.185+0000 7f691dd2b640 1 --2- 172.21.15.37:0/1753090106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69180d0ff0 0x7f69180ce470 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.185+0000 7f691dd2b640 1 -- 172.21.15.37:0/1753090106 >> 172.21.15.37:0/1753090106 conn(0x7f691806de00 msgr2=0x7f691815e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:04.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.185+0000 7f691dd2b640 1 -- 172.21.15.37:0/1753090106 shutdown_connections 2024-09-17T04:34:04.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.185+0000 7f691dd2b640 1 -- 172.21.15.37:0/1753090106 wait complete. 2024-09-17T04:34:04.196 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2024-09-17T04:34:02.456006+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":144,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220364,"kb_used_data":4104,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754356,"statfs":{"total":773094113280,"available":772868460544,"internally_reserved":0,"allocated":4202496,"data_stored":2322356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.363041"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.425136+0000","last_change":"2024-09-17T04:34:00.133489+0000","last_active":"2024-09-17T04:34:01.425136+0000","last_peered":"2024-09-17T04:34:01.425136+0000","last_clean":"2024-09-17T04:34:01.425136+0000","last_became_active":"2024-09-17T04:34:00.133172+0000","last_became_peered":"2024-09-17T04:34:00.133172+0000","last_unstale":"2024-09-17T04:34:01.425136+0000","last_undegraded":"2024-09-17T04:34:01.425136+0000","last_fullsized":"2024-09-17T04:34:01.425136+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:34.594760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134534+0000","last_change":"2024-09-17T04:34:00.141408+0000","last_active":"2024-09-17T04:34:01.134534+0000","last_peered":"2024-09-17T04:34:01.134534+0000","last_clean":"2024-09-17T04:34:01.134534+0000","last_became_active":"2024-09-17T04:34:00.141077+0000","last_became_peered":"2024-09-17T04:34:00.141077+0000","last_unstale":"2024-09-17T04:34:01.134534+0000","last_undegraded":"2024-09-17T04:34:01.134534+0000","last_fullsized":"2024-09-17T04:34:01.134534+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:24:35.627644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.536143+0000","last_change":"2024-09-17T04:34:00.146084+0000","last_active":"2024-09-17T04:34:01.536143+0000","last_peered":"2024-09-17T04:34:01.536143+0000","last_clean":"2024-09-17T04:34:01.536143+0000","last_became_active":"2024-09-17T04:34:00.145314+0000","last_became_peered":"2024-09-17T04:34:00.145314+0000","last_unstale":"2024-09-17T04:34:01.536143+0000","last_undegraded":"2024-09-17T04:34:01.536143+0000","last_fullsized":"2024-09-17T04:34:01.536143+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:25.874838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542085+0000","last_change":"2024-09-17T04:34:00.138845+0000","last_active":"2024-09-17T04:34:01.542085+0000","last_peered":"2024-09-17T04:34:01.542085+0000","last_clean":"2024-09-17T04:34:01.542085+0000","last_became_active":"2024-09-17T04:34:00.138517+0000","last_became_peered":"2024-09-17T04:34:00.138517+0000","last_unstale":"2024-09-17T04:34:01.542085+0000","last_undegraded":"2024-09-17T04:34:01.542085+0000","last_fullsized":"2024-09-17T04:34:01.542085+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:17.893360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.133673+0000","last_change":"2024-09-17T04:34:00.148227+0000","last_active":"2024-09-17T04:34:01.133673+0000","last_peered":"2024-09-17T04:34:01.133673+0000","last_clean":"2024-09-17T04:34:01.133673+0000","last_became_active":"2024-09-17T04:34:00.147902+0000","last_became_peered":"2024-09-17T04:34:00.147902+0000","last_unstale":"2024-09-17T04:34:01.133673+0000","last_undegraded":"2024-09-17T04:34:01.133673+0000","last_fullsized":"2024-09-17T04:34:01.133673+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:52.789669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134277+0000","last_change":"2024-09-17T04:34:00.139638+0000","last_active":"2024-09-17T04:34:01.134277+0000","last_peered":"2024-09-17T04:34:01.134277+0000","last_clean":"2024-09-17T04:34:01.134277+0000","last_became_active":"2024-09-17T04:34:00.139252+0000","last_became_peered":"2024-09-17T04:34:00.139252+0000","last_unstale":"2024-09-17T04:34:01.134277+0000","last_undegraded":"2024-09-17T04:34:01.134277+0000","last_fullsized":"2024-09-17T04:34:01.134277+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:42.085058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542050+0000","last_change":"2024-09-17T04:34:00.135963+0000","last_active":"2024-09-17T04:34:01.542050+0000","last_peered":"2024-09-17T04:34:01.542050+0000","last_clean":"2024-09-17T04:34:01.542050+0000","last_became_active":"2024-09-17T04:34:00.135466+0000","last_became_peered":"2024-09-17T04:34:00.135466+0000","last_unstale":"2024-09-17T04:34:01.542050+0000","last_undegraded":"2024-09-17T04:34:01.542050+0000","last_fullsized":"2024-09-17T04:34:01.542050+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:49:15.290254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.541777+0000","last_change":"2024-09-17T04:34:00.136461+0000","last_active":"2024-09-17T04:34:01.541777+0000","last_peered":"2024-09-17T04:34:01.541777+0000","last_clean":"2024-09-17T04:34:01.541777+0000","last_became_active":"2024-09-17T04:34:00.136127+0000","last_became_peered":"2024-09-17T04:34:00.136127+0000","last_unstale":"2024-09-17T04:34:01.541777+0000","last_undegraded":"2024-09-17T04:34:01.541777+0000","last_fullsized":"2024-09-17T04:34:01.541777+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:57:08.760910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.135030+0000","last_change":"2024-09-17T04:34:00.126244+0000","last_active":"2024-09-17T04:34:01.135030+0000","last_peered":"2024-09-17T04:34:01.135030+0000","last_clean":"2024-09-17T04:34:01.135030+0000","last_became_active":"2024-09-17T04:34:00.125921+0000","last_became_peered":"2024-09-17T04:34:00.125921+0000","last_unstale":"2024-09-17T04:34:01.135030+0000","last_undegraded":"2024-09-17T04:34:01.135030+0000","last_fullsized":"2024-09-17T04:34:01.135030+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:21:47.588781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542872+0000","last_change":"2024-09-17T04:34:00.147614+0000","last_active":"2024-09-17T04:34:01.542872+0000","last_peered":"2024-09-17T04:34:01.542872+0000","last_clean":"2024-09-17T04:34:01.542872+0000","last_became_active":"2024-09-17T04:34:00.147198+0000","last_became_peered":"2024-09-17T04:34:00.147198+0000","last_unstale":"2024-09-17T04:34:01.542872+0000","last_undegraded":"2024-09-17T04:34:01.542872+0000","last_fullsized":"2024-09-17T04:34:01.542872+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:02.203052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542113+0000","last_change":"2024-09-17T04:34:00.144759+0000","last_active":"2024-09-17T04:34:01.542113+0000","last_peered":"2024-09-17T04:34:01.542113+0000","last_clean":"2024-09-17T04:34:01.542113+0000","last_became_active":"2024-09-17T04:34:00.144463+0000","last_became_peered":"2024-09-17T04:34:00.144463+0000","last_unstale":"2024-09-17T04:34:01.542113+0000","last_undegraded":"2024-09-17T04:34:01.542113+0000","last_fullsized":"2024-09-17T04:34:01.542113+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:09:46.692534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.536185+0000","last_change":"2024-09-17T04:34:00.144495+0000","last_active":"2024-09-17T04:34:01.536185+0000","last_peered":"2024-09-17T04:34:01.536185+0000","last_clean":"2024-09-17T04:34:01.536185+0000","last_became_active":"2024-09-17T04:34:00.144027+0000","last_became_peered":"2024-09-17T04:34:00.144027+0000","last_unstale":"2024-09-17T04:34:01.536185+0000","last_undegraded":"2024-09-17T04:34:01.536185+0000","last_fullsized":"2024-09-17T04:34:01.536185+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:09:46.692534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.133926+0000","last_change":"2024-09-17T04:34:00.137663+0000","last_active":"2024-09-17T04:34:01.133926+0000","last_peered":"2024-09-17T04:34:01.133926+0000","last_clean":"2024-09-17T04:34:01.133926+0000","last_became_active":"2024-09-17T04:34:00.137306+0000","last_became_peered":"2024-09-17T04:34:00.137306+0000","last_unstale":"2024-09-17T04:34:01.133926+0000","last_undegraded":"2024-09-17T04:34:01.133926+0000","last_fullsized":"2024-09-17T04:34:01.133926+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:24.590166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134622+0000","last_change":"2024-09-17T04:34:00.140479+0000","last_active":"2024-09-17T04:34:01.134622+0000","last_peered":"2024-09-17T04:34:01.134622+0000","last_clean":"2024-09-17T04:34:01.134622+0000","last_became_active":"2024-09-17T04:34:00.140125+0000","last_became_peered":"2024-09-17T04:34:00.140125+0000","last_unstale":"2024-09-17T04:34:01.134622+0000","last_undegraded":"2024-09-17T04:34:01.134622+0000","last_fullsized":"2024-09-17T04:34:01.134622+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:05.573033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.535981+0000","last_change":"2024-09-17T04:34:00.135929+0000","last_active":"2024-09-17T04:34:01.535981+0000","last_peered":"2024-09-17T04:34:01.535981+0000","last_clean":"2024-09-17T04:34:01.535981+0000","last_became_active":"2024-09-17T04:34:00.135615+0000","last_became_peered":"2024-09-17T04:34:00.135615+0000","last_unstale":"2024-09-17T04:34:01.535981+0000","last_undegraded":"2024-09-17T04:34:01.535981+0000","last_fullsized":"2024-09-17T04:34:01.535981+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:47:51.639001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542164+0000","last_change":"2024-09-17T04:34:00.147088+0000","last_active":"2024-09-17T04:34:01.542164+0000","last_peered":"2024-09-17T04:34:01.542164+0000","last_clean":"2024-09-17T04:34:01.542164+0000","last_became_active":"2024-09-17T04:34:00.146778+0000","last_became_peered":"2024-09-17T04:34:00.146778+0000","last_unstale":"2024-09-17T04:34:01.542164+0000","last_undegraded":"2024-09-17T04:34:01.542164+0000","last_fullsized":"2024-09-17T04:34:01.542164+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:38.015410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542362+0000","last_change":"2024-09-17T04:34:00.134671+0000","last_active":"2024-09-17T04:34:01.542362+0000","last_peered":"2024-09-17T04:34:01.542362+0000","last_clean":"2024-09-17T04:34:01.542362+0000","last_became_active":"2024-09-17T04:34:00.132323+0000","last_became_peered":"2024-09-17T04:34:00.132323+0000","last_unstale":"2024-09-17T04:34:01.542362+0000","last_undegraded":"2024-09-17T04:34:01.542362+0000","last_fullsized":"2024-09-17T04:34:01.542362+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:58.534910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134576+0000","last_change":"2024-09-17T04:34:00.147773+0000","last_active":"2024-09-17T04:34:01.134576+0000","last_peered":"2024-09-17T04:34:01.134576+0000","last_clean":"2024-09-17T04:34:01.134576+0000","last_became_active":"2024-09-17T04:34:00.147077+0000","last_became_peered":"2024-09-17T04:34:00.147077+0000","last_unstale":"2024-09-17T04:34:01.134576+0000","last_undegraded":"2024-09-17T04:34:01.134576+0000","last_fullsized":"2024-09-17T04:34:01.134576+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:04:29.231102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.133935+0000","last_change":"2024-09-17T04:34:00.149546+0000","last_active":"2024-09-17T04:34:01.133935+0000","last_peered":"2024-09-17T04:34:01.133935+0000","last_clean":"2024-09-17T04:34:01.133935+0000","last_became_active":"2024-09-17T04:34:00.149166+0000","last_became_peered":"2024-09-17T04:34:00.149166+0000","last_unstale":"2024-09-17T04:34:01.133935+0000","last_undegraded":"2024-09-17T04:34:01.133935+0000","last_fullsized":"2024-09-17T04:34:01.133935+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:09:46.692534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134934+0000","last_change":"2024-09-17T04:34:00.140053+0000","last_active":"2024-09-17T04:34:01.134934+0000","last_peered":"2024-09-17T04:34:01.134934+0000","last_clean":"2024-09-17T04:34:01.134934+0000","last_became_active":"2024-09-17T04:34:00.139557+0000","last_became_peered":"2024-09-17T04:34:00.139557+0000","last_unstale":"2024-09-17T04:34:01.134934+0000","last_undegraded":"2024-09-17T04:34:01.134934+0000","last_fullsized":"2024-09-17T04:34:01.134934+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:50:59.831340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134657+0000","last_change":"2024-09-17T04:34:00.133295+0000","last_active":"2024-09-17T04:34:01.134657+0000","last_peered":"2024-09-17T04:34:01.134657+0000","last_clean":"2024-09-17T04:34:01.134657+0000","last_became_active":"2024-09-17T04:34:00.132965+0000","last_became_peered":"2024-09-17T04:34:00.132965+0000","last_unstale":"2024-09-17T04:34:01.134657+0000","last_undegraded":"2024-09-17T04:34:01.134657+0000","last_fullsized":"2024-09-17T04:34:01.134657+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:52.024315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134610+0000","last_change":"2024-09-17T04:34:00.148488+0000","last_active":"2024-09-17T04:34:01.134610+0000","last_peered":"2024-09-17T04:34:01.134610+0000","last_clean":"2024-09-17T04:34:01.134610+0000","last_became_active":"2024-09-17T04:34:00.148176+0000","last_became_peered":"2024-09-17T04:34:00.148176+0000","last_unstale":"2024-09-17T04:34:01.134610+0000","last_undegraded":"2024-09-17T04:34:01.134610+0000","last_fullsized":"2024-09-17T04:34:01.134610+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:23.552526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.541908+0000","last_change":"2024-09-17T04:34:00.132040+0000","last_active":"2024-09-17T04:34:01.541908+0000","last_peered":"2024-09-17T04:34:01.541908+0000","last_clean":"2024-09-17T04:34:01.541908+0000","last_became_active":"2024-09-17T04:34:00.131717+0000","last_became_peered":"2024-09-17T04:34:00.131717+0000","last_unstale":"2024-09-17T04:34:01.541908+0000","last_undegraded":"2024-09-17T04:34:01.541908+0000","last_fullsized":"2024-09-17T04:34:01.541908+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:27.558398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.535779+0000","last_change":"2024-09-17T04:34:00.137953+0000","last_active":"2024-09-17T04:34:01.535779+0000","last_peered":"2024-09-17T04:34:01.535779+0000","last_clean":"2024-09-17T04:34:01.535779+0000","last_became_active":"2024-09-17T04:34:00.137577+0000","last_became_peered":"2024-09-17T04:34:00.137577+0000","last_unstale":"2024-09-17T04:34:01.535779+0000","last_undegraded":"2024-09-17T04:34:01.535779+0000","last_fullsized":"2024-09-17T04:34:01.535779+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:33:39.300267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542955+0000","last_change":"2024-09-17T04:34:00.134911+0000","last_active":"2024-09-17T04:34:01.542955+0000","last_peered":"2024-09-17T04:34:01.542955+0000","last_clean":"2024-09-17T04:34:01.542955+0000","last_became_active":"2024-09-17T04:34:00.133373+0000","last_became_peered":"2024-09-17T04:34:00.133373+0000","last_unstale":"2024-09-17T04:34:01.542955+0000","last_undegraded":"2024-09-17T04:34:01.542955+0000","last_fullsized":"2024-09-17T04:34:01.542955+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:27:05.531961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134763+0000","last_change":"2024-09-17T04:34:00.147296+0000","last_active":"2024-09-17T04:34:01.134763+0000","last_peered":"2024-09-17T04:34:01.134763+0000","last_clean":"2024-09-17T04:34:01.134763+0000","last_became_active":"2024-09-17T04:34:00.146984+0000","last_became_peered":"2024-09-17T04:34:00.146984+0000","last_unstale":"2024-09-17T04:34:01.134763+0000","last_undegraded":"2024-09-17T04:34:01.134763+0000","last_fullsized":"2024-09-17T04:34:01.134763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:51.136426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542793+0000","last_change":"2024-09-17T04:34:00.136351+0000","last_active":"2024-09-17T04:34:01.542793+0000","last_peered":"2024-09-17T04:34:01.542793+0000","last_clean":"2024-09-17T04:34:01.542793+0000","last_became_active":"2024-09-17T04:34:00.136007+0000","last_became_peered":"2024-09-17T04:34:00.136007+0000","last_unstale":"2024-09-17T04:34:01.542793+0000","last_undegraded":"2024-09-17T04:34:01.542793+0000","last_fullsized":"2024-09-17T04:34:01.542793+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:37:23.029519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.536329+0000","last_change":"2024-09-17T04:34:00.134272+0000","last_active":"2024-09-17T04:34:01.536329+0000","last_peered":"2024-09-17T04:34:01.536329+0000","last_clean":"2024-09-17T04:34:01.536329+0000","last_became_active":"2024-09-17T04:34:00.133409+0000","last_became_peered":"2024-09-17T04:34:00.133409+0000","last_unstale":"2024-09-17T04:34:01.536329+0000","last_undegraded":"2024-09-17T04:34:01.536329+0000","last_fullsized":"2024-09-17T04:34:01.536329+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:38.015410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134538+0000","last_change":"2024-09-17T04:34:00.146595+0000","last_active":"2024-09-17T04:34:01.134538+0000","last_peered":"2024-09-17T04:34:01.134538+0000","last_clean":"2024-09-17T04:34:01.134538+0000","last_became_active":"2024-09-17T04:34:00.145812+0000","last_became_peered":"2024-09-17T04:34:00.145812+0000","last_unstale":"2024-09-17T04:34:01.134538+0000","last_undegraded":"2024-09-17T04:34:01.134538+0000","last_fullsized":"2024-09-17T04:34:01.134538+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:15.949787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.541869+0000","last_change":"2024-09-17T04:34:00.131809+0000","last_active":"2024-09-17T04:34:01.541869+0000","last_peered":"2024-09-17T04:34:01.541869+0000","last_clean":"2024-09-17T04:34:01.541869+0000","last_became_active":"2024-09-17T04:34:00.131532+0000","last_became_peered":"2024-09-17T04:34:00.131532+0000","last_unstale":"2024-09-17T04:34:01.541869+0000","last_undegraded":"2024-09-17T04:34:01.541869+0000","last_fullsized":"2024-09-17T04:34:01.541869+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:25:50.372117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.425144+0000","last_change":"2024-09-17T04:34:00.140013+0000","last_active":"2024-09-17T04:34:01.425144+0000","last_peered":"2024-09-17T04:34:01.425144+0000","last_clean":"2024-09-17T04:34:01.425144+0000","last_became_active":"2024-09-17T04:34:00.139703+0000","last_became_peered":"2024-09-17T04:34:00.139703+0000","last_unstale":"2024-09-17T04:34:01.425144+0000","last_undegraded":"2024-09-17T04:34:01.425144+0000","last_fullsized":"2024-09-17T04:34:01.425144+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:59:37.912219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.541803+0000","last_change":"2024-09-17T04:34:00.148182+0000","last_active":"2024-09-17T04:34:01.541803+0000","last_peered":"2024-09-17T04:34:01.541803+0000","last_clean":"2024-09-17T04:34:01.541803+0000","last_became_active":"2024-09-17T04:34:00.147537+0000","last_became_peered":"2024-09-17T04:34:00.147537+0000","last_unstale":"2024-09-17T04:34:01.541803+0000","last_undegraded":"2024-09-17T04:34:01.541803+0000","last_fullsized":"2024-09-17T04:34:01.541803+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:37:23.029519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542456+0000","last_change":"2024-09-17T04:34:00.139991+0000","last_active":"2024-09-17T04:34:01.542456+0000","last_peered":"2024-09-17T04:34:01.542456+0000","last_clean":"2024-09-17T04:34:01.542456+0000","last_became_active":"2024-09-17T04:34:00.139640+0000","last_became_peered":"2024-09-17T04:34:00.139640+0000","last_unstale":"2024-09-17T04:34:01.542456+0000","last_undegraded":"2024-09-17T04:34:01.542456+0000","last_fullsized":"2024-09-17T04:34:01.542456+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:57:02.382202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.425229+0000","last_change":"2024-09-17T04:33:59.113533+0000","last_active":"2024-09-17T04:34:01.425229+0000","last_peered":"2024-09-17T04:34:01.425229+0000","last_clean":"2024-09-17T04:34:01.425229+0000","last_became_active":"2024-09-17T04:33:59.113062+0000","last_became_peered":"2024-09-17T04:33:59.113062+0000","last_unstale":"2024-09-17T04:34:01.425229+0000","last_undegraded":"2024-09-17T04:34:01.425229+0000","last_fullsized":"2024-09-17T04:34:01.425229+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:00.673571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.425212+0000","last_change":"2024-09-17T04:34:00.143656+0000","last_active":"2024-09-17T04:34:01.425212+0000","last_peered":"2024-09-17T04:34:01.425212+0000","last_clean":"2024-09-17T04:34:01.425212+0000","last_became_active":"2024-09-17T04:34:00.143327+0000","last_became_peered":"2024-09-17T04:34:00.143327+0000","last_unstale":"2024-09-17T04:34:01.425212+0000","last_undegraded":"2024-09-17T04:34:01.425212+0000","last_fullsized":"2024-09-17T04:34:01.425212+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:02:15.227761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542587+0000","last_change":"2024-09-17T04:33:59.113364+0000","last_active":"2024-09-17T04:34:01.542587+0000","last_peered":"2024-09-17T04:34:01.542587+0000","last_clean":"2024-09-17T04:34:01.542587+0000","last_became_active":"2024-09-17T04:33:59.112021+0000","last_became_peered":"2024-09-17T04:33:59.112021+0000","last_unstale":"2024-09-17T04:34:01.542587+0000","last_undegraded":"2024-09-17T04:34:01.542587+0000","last_fullsized":"2024-09-17T04:34:01.542587+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:53:00.824171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134850+0000","last_change":"2024-09-17T04:34:00.148658+0000","last_active":"2024-09-17T04:34:01.134850+0000","last_peered":"2024-09-17T04:34:01.134850+0000","last_clean":"2024-09-17T04:34:01.134850+0000","last_became_active":"2024-09-17T04:34:00.148350+0000","last_became_peered":"2024-09-17T04:34:00.148350+0000","last_unstale":"2024-09-17T04:34:01.134850+0000","last_undegraded":"2024-09-17T04:34:01.134850+0000","last_fullsized":"2024-09-17T04:34:01.134850+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:25:19.913461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134108+0000","last_change":"2024-09-17T04:33:59.124080+0000","last_active":"2024-09-17T04:34:01.134108+0000","last_peered":"2024-09-17T04:34:01.134108+0000","last_clean":"2024-09-17T04:34:01.134108+0000","last_became_active":"2024-09-17T04:33:59.123624+0000","last_became_peered":"2024-09-17T04:33:59.123624+0000","last_unstale":"2024-09-17T04:34:01.134108+0000","last_undegraded":"2024-09-17T04:34:01.134108+0000","last_fullsized":"2024-09-17T04:34:01.134108+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:52.054448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542775+0000","last_change":"2024-09-17T04:34:00.136203+0000","last_active":"2024-09-17T04:34:01.542775+0000","last_peered":"2024-09-17T04:34:01.542775+0000","last_clean":"2024-09-17T04:34:01.542775+0000","last_became_active":"2024-09-17T04:34:00.135871+0000","last_became_peered":"2024-09-17T04:34:00.135871+0000","last_unstale":"2024-09-17T04:34:01.542775+0000","last_undegraded":"2024-09-17T04:34:01.542775+0000","last_fullsized":"2024-09-17T04:34:01.542775+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:41.010278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134892+0000","last_change":"2024-09-17T04:33:59.119452+0000","last_active":"2024-09-17T04:34:01.134892+0000","last_peered":"2024-09-17T04:34:01.134892+0000","last_clean":"2024-09-17T04:34:01.134892+0000","last_became_active":"2024-09-17T04:33:59.119108+0000","last_became_peered":"2024-09-17T04:33:59.119108+0000","last_unstale":"2024-09-17T04:34:01.134892+0000","last_undegraded":"2024-09-17T04:34:01.134892+0000","last_fullsized":"2024-09-17T04:34:01.134892+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:52.325898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.541792+0000","last_change":"2024-09-17T04:34:00.143488+0000","last_active":"2024-09-17T04:34:01.541792+0000","last_peered":"2024-09-17T04:34:01.541792+0000","last_clean":"2024-09-17T04:34:01.541792+0000","last_became_active":"2024-09-17T04:34:00.143166+0000","last_became_peered":"2024-09-17T04:34:00.143166+0000","last_unstale":"2024-09-17T04:34:01.541792+0000","last_undegraded":"2024-09-17T04:34:01.541792+0000","last_fullsized":"2024-09-17T04:34:01.541792+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:27:05.531961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134677+0000","last_change":"2024-09-17T04:33:59.114365+0000","last_active":"2024-09-17T04:34:01.134677+0000","last_peered":"2024-09-17T04:34:01.134677+0000","last_clean":"2024-09-17T04:34:01.134677+0000","last_became_active":"2024-09-17T04:33:59.114058+0000","last_became_peered":"2024-09-17T04:33:59.114058+0000","last_unstale":"2024-09-17T04:34:01.134677+0000","last_undegraded":"2024-09-17T04:34:01.134677+0000","last_fullsized":"2024-09-17T04:34:01.134677+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:43:41.290086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.536148+0000","last_change":"2024-09-17T04:34:00.142714+0000","last_active":"2024-09-17T04:34:01.536148+0000","last_peered":"2024-09-17T04:34:01.536148+0000","last_clean":"2024-09-17T04:34:01.536148+0000","last_became_active":"2024-09-17T04:34:00.142287+0000","last_became_peered":"2024-09-17T04:34:00.142287+0000","last_unstale":"2024-09-17T04:34:01.536148+0000","last_undegraded":"2024-09-17T04:34:01.536148+0000","last_fullsized":"2024-09-17T04:34:01.536148+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:49:15.290254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134401+0000","last_change":"2024-09-17T04:33:59.117052+0000","last_active":"2024-09-17T04:34:01.134401+0000","last_peered":"2024-09-17T04:34:01.134401+0000","last_clean":"2024-09-17T04:34:01.134401+0000","last_became_active":"2024-09-17T04:33:59.116738+0000","last_became_peered":"2024-09-17T04:33:59.116738+0000","last_unstale":"2024-09-17T04:34:01.134401+0000","last_undegraded":"2024-09-17T04:34:01.134401+0000","last_fullsized":"2024-09-17T04:34:01.134401+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:09.152247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134181+0000","last_change":"2024-09-17T04:34:00.132375+0000","last_active":"2024-09-17T04:34:01.134181+0000","last_peered":"2024-09-17T04:34:01.134181+0000","last_clean":"2024-09-17T04:34:01.134181+0000","last_became_active":"2024-09-17T04:34:00.131490+0000","last_became_peered":"2024-09-17T04:34:00.131490+0000","last_unstale":"2024-09-17T04:34:01.134181+0000","last_undegraded":"2024-09-17T04:34:01.134181+0000","last_fullsized":"2024-09-17T04:34:01.134181+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:56.019994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134132+0000","last_change":"2024-09-17T04:33:59.121968+0000","last_active":"2024-09-17T04:34:01.134132+0000","last_peered":"2024-09-17T04:34:01.134132+0000","last_clean":"2024-09-17T04:34:01.134132+0000","last_became_active":"2024-09-17T04:33:59.121152+0000","last_became_peered":"2024-09-17T04:33:59.121152+0000","last_unstale":"2024-09-17T04:34:01.134132+0000","last_undegraded":"2024-09-17T04:34:01.134132+0000","last_fullsized":"2024-09-17T04:34:01.134132+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:02:14.225662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.541965+0000","last_change":"2024-09-17T04:34:00.137017+0000","last_active":"2024-09-17T04:34:01.541965+0000","last_peered":"2024-09-17T04:34:01.541965+0000","last_clean":"2024-09-17T04:34:01.541965+0000","last_became_active":"2024-09-17T04:34:00.136469+0000","last_became_peered":"2024-09-17T04:34:00.136469+0000","last_unstale":"2024-09-17T04:34:01.541965+0000","last_undegraded":"2024-09-17T04:34:01.541965+0000","last_fullsized":"2024-09-17T04:34:01.541965+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:39:50.199228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134056+0000","last_change":"2024-09-17T04:33:59.109225+0000","last_active":"2024-09-17T04:34:01.134056+0000","last_peered":"2024-09-17T04:34:01.134056+0000","last_clean":"2024-09-17T04:34:01.134056+0000","last_became_active":"2024-09-17T04:33:59.108890+0000","last_became_peered":"2024-09-17T04:33:59.108890+0000","last_unstale":"2024-09-17T04:34:01.134056+0000","last_undegraded":"2024-09-17T04:34:01.134056+0000","last_fullsized":"2024-09-17T04:34:01.134056+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:59.447364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.536396+0000","last_change":"2024-09-17T04:34:00.149518+0000","last_active":"2024-09-17T04:34:01.536396+0000","last_peered":"2024-09-17T04:34:01.536396+0000","last_clean":"2024-09-17T04:34:01.536396+0000","last_became_active":"2024-09-17T04:34:00.149133+0000","last_became_peered":"2024-09-17T04:34:00.149133+0000","last_unstale":"2024-09-17T04:34:01.536396+0000","last_undegraded":"2024-09-17T04:34:01.536396+0000","last_fullsized":"2024-09-17T04:34:01.536396+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:09:37.721140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134378+0000","last_change":"2024-09-17T04:33:59.120720+0000","last_active":"2024-09-17T04:34:01.134378+0000","last_peered":"2024-09-17T04:34:01.134378+0000","last_clean":"2024-09-17T04:34:01.134378+0000","last_became_active":"2024-09-17T04:33:59.120386+0000","last_became_peered":"2024-09-17T04:33:59.120386+0000","last_unstale":"2024-09-17T04:34:01.134378+0000","last_undegraded":"2024-09-17T04:34:01.134378+0000","last_fullsized":"2024-09-17T04:34:01.134378+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:43.682967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134069+0000","last_change":"2024-09-17T04:34:00.150547+0000","last_active":"2024-09-17T04:34:01.134069+0000","last_peered":"2024-09-17T04:34:01.134069+0000","last_clean":"2024-09-17T04:34:01.134069+0000","last_became_active":"2024-09-17T04:34:00.146052+0000","last_became_peered":"2024-09-17T04:34:00.146052+0000","last_unstale":"2024-09-17T04:34:01.134069+0000","last_undegraded":"2024-09-17T04:34:01.134069+0000","last_fullsized":"2024-09-17T04:34:01.134069+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:22:12.941844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542591+0000","last_change":"2024-09-17T04:33:59.112992+0000","last_active":"2024-09-17T04:34:01.542591+0000","last_peered":"2024-09-17T04:34:01.542591+0000","last_clean":"2024-09-17T04:34:01.542591+0000","last_became_active":"2024-09-17T04:33:59.112657+0000","last_became_peered":"2024-09-17T04:33:59.112657+0000","last_unstale":"2024-09-17T04:34:01.542591+0000","last_undegraded":"2024-09-17T04:34:01.542591+0000","last_fullsized":"2024-09-17T04:34:01.542591+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:06.203780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134425+0000","last_change":"2024-09-17T04:34:00.148749+0000","last_active":"2024-09-17T04:34:01.134425+0000","last_peered":"2024-09-17T04:34:01.134425+0000","last_clean":"2024-09-17T04:34:01.134425+0000","last_became_active":"2024-09-17T04:34:00.147951+0000","last_became_peered":"2024-09-17T04:34:00.147951+0000","last_unstale":"2024-09-17T04:34:01.134425+0000","last_undegraded":"2024-09-17T04:34:01.134425+0000","last_fullsized":"2024-09-17T04:34:01.134425+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:46.711607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.536289+0000","last_change":"2024-09-17T04:33:59.112913+0000","last_active":"2024-09-17T04:34:01.536289+0000","last_peered":"2024-09-17T04:34:01.536289+0000","last_clean":"2024-09-17T04:34:01.536289+0000","last_became_active":"2024-09-17T04:33:59.112540+0000","last_became_peered":"2024-09-17T04:33:59.112540+0000","last_unstale":"2024-09-17T04:34:01.536289+0000","last_undegraded":"2024-09-17T04:34:01.536289+0000","last_fullsized":"2024-09-17T04:34:01.536289+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:33.592661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.536113+0000","last_change":"2024-09-17T04:34:00.146174+0000","last_active":"2024-09-17T04:34:01.536113+0000","last_peered":"2024-09-17T04:34:01.536113+0000","last_clean":"2024-09-17T04:34:01.536113+0000","last_became_active":"2024-09-17T04:34:00.145752+0000","last_became_peered":"2024-09-17T04:34:00.145752+0000","last_unstale":"2024-09-17T04:34:01.536113+0000","last_undegraded":"2024-09-17T04:34:01.536113+0000","last_fullsized":"2024-09-17T04:34:01.536113+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:44.352160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134384+0000","last_change":"2024-09-17T04:33:59.124191+0000","last_active":"2024-09-17T04:34:01.134384+0000","last_peered":"2024-09-17T04:34:01.134384+0000","last_clean":"2024-09-17T04:34:01.134384+0000","last_became_active":"2024-09-17T04:33:59.123823+0000","last_became_peered":"2024-09-17T04:33:59.123823+0000","last_unstale":"2024-09-17T04:34:01.134384+0000","last_undegraded":"2024-09-17T04:34:01.134384+0000","last_fullsized":"2024-09-17T04:34:01.134384+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:39:49.197130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542212+0000","last_change":"2024-09-17T04:34:00.132724+0000","last_active":"2024-09-17T04:34:01.542212+0000","last_peered":"2024-09-17T04:34:01.542212+0000","last_clean":"2024-09-17T04:34:01.542212+0000","last_became_active":"2024-09-17T04:34:00.131882+0000","last_became_peered":"2024-09-17T04:34:00.131882+0000","last_unstale":"2024-09-17T04:34:01.542212+0000","last_undegraded":"2024-09-17T04:34:01.542212+0000","last_fullsized":"2024-09-17T04:34:01.542212+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:48.008056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134860+0000","last_change":"2024-09-17T04:33:59.109143+0000","last_active":"2024-09-17T04:34:01.134860+0000","last_peered":"2024-09-17T04:34:01.134860+0000","last_clean":"2024-09-17T04:34:01.134860+0000","last_became_active":"2024-09-17T04:33:59.108847+0000","last_became_peered":"2024-09-17T04:33:59.108847+0000","last_unstale":"2024-09-17T04:34:01.134860+0000","last_undegraded":"2024-09-17T04:34:01.134860+0000","last_fullsized":"2024-09-17T04:34:01.134860+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:18.019734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.133940+0000","last_change":"2024-09-17T04:34:00.145327+0000","last_active":"2024-09-17T04:34:01.133940+0000","last_peered":"2024-09-17T04:34:01.133940+0000","last_clean":"2024-09-17T04:34:01.133940+0000","last_became_active":"2024-09-17T04:34:00.142809+0000","last_became_peered":"2024-09-17T04:34:00.142809+0000","last_unstale":"2024-09-17T04:34:01.133940+0000","last_undegraded":"2024-09-17T04:34:01.133940+0000","last_fullsized":"2024-09-17T04:34:01.133940+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:47:41.427853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542157+0000","last_change":"2024-09-17T04:33:59.116224+0000","last_active":"2024-09-17T04:34:01.542157+0000","last_peered":"2024-09-17T04:34:01.542157+0000","last_clean":"2024-09-17T04:34:01.542157+0000","last_became_active":"2024-09-17T04:33:59.115862+0000","last_became_peered":"2024-09-17T04:33:59.115862+0000","last_unstale":"2024-09-17T04:34:01.542157+0000","last_undegraded":"2024-09-17T04:34:01.542157+0000","last_fullsized":"2024-09-17T04:34:01.542157+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:22:13.092980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542180+0000","last_change":"2024-09-17T04:34:00.148329+0000","last_active":"2024-09-17T04:34:01.542180+0000","last_peered":"2024-09-17T04:34:01.542180+0000","last_clean":"2024-09-17T04:34:01.542180+0000","last_became_active":"2024-09-17T04:34:00.147988+0000","last_became_peered":"2024-09-17T04:34:00.147988+0000","last_unstale":"2024-09-17T04:34:01.542180+0000","last_undegraded":"2024-09-17T04:34:01.542180+0000","last_fullsized":"2024-09-17T04:34:01.542180+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:09:37.721140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134261+0000","last_change":"2024-09-17T04:33:59.120796+0000","last_active":"2024-09-17T04:34:01.134261+0000","last_peered":"2024-09-17T04:34:01.134261+0000","last_clean":"2024-09-17T04:34:01.134261+0000","last_became_active":"2024-09-17T04:33:59.120478+0000","last_became_peered":"2024-09-17T04:33:59.120478+0000","last_unstale":"2024-09-17T04:34:01.134261+0000","last_undegraded":"2024-09-17T04:34:01.134261+0000","last_fullsized":"2024-09-17T04:34:01.134261+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:54.677479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.425130+0000","last_change":"2024-09-17T04:34:00.147713+0000","last_active":"2024-09-17T04:34:01.425130+0000","last_peered":"2024-09-17T04:34:01.425130+0000","last_clean":"2024-09-17T04:34:01.425130+0000","last_became_active":"2024-09-17T04:34:00.147099+0000","last_became_peered":"2024-09-17T04:34:00.147099+0000","last_unstale":"2024-09-17T04:34:01.425130+0000","last_undegraded":"2024-09-17T04:34:01.425130+0000","last_fullsized":"2024-09-17T04:34:01.425130+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:22:45.296651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134091+0000","last_change":"2024-09-17T04:33:59.122217+0000","last_active":"2024-09-17T04:34:01.134091+0000","last_peered":"2024-09-17T04:34:01.134091+0000","last_clean":"2024-09-17T04:34:01.134091+0000","last_became_active":"2024-09-17T04:33:59.121866+0000","last_became_peered":"2024-09-17T04:33:59.121866+0000","last_unstale":"2024-09-17T04:34:01.134091+0000","last_undegraded":"2024-09-17T04:34:01.134091+0000","last_fullsized":"2024-09-17T04:34:01.134091+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:33.592661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134630+0000","last_change":"2024-09-17T04:34:00.140363+0000","last_active":"2024-09-17T04:34:01.134630+0000","last_peered":"2024-09-17T04:34:01.134630+0000","last_clean":"2024-09-17T04:34:01.134630+0000","last_became_active":"2024-09-17T04:34:00.138809+0000","last_became_peered":"2024-09-17T04:34:00.138809+0000","last_unstale":"2024-09-17T04:34:01.134630+0000","last_undegraded":"2024-09-17T04:34:01.134630+0000","last_fullsized":"2024-09-17T04:34:01.134630+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:50.605107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542965+0000","last_change":"2024-09-17T04:33:59.118082+0000","last_active":"2024-09-17T04:34:01.542965+0000","last_peered":"2024-09-17T04:34:01.542965+0000","last_clean":"2024-09-17T04:34:01.542965+0000","last_became_active":"2024-09-17T04:33:59.117749+0000","last_became_peered":"2024-09-17T04:33:59.117749+0000","last_unstale":"2024-09-17T04:34:01.542965+0000","last_undegraded":"2024-09-17T04:34:01.542965+0000","last_fullsized":"2024-09-17T04:34:01.542965+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:22:09.267634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.536532+0000","last_change":"2024-09-17T04:34:00.143589+0000","last_active":"2024-09-17T04:34:01.536532+0000","last_peered":"2024-09-17T04:34:01.536532+0000","last_clean":"2024-09-17T04:34:01.536532+0000","last_became_active":"2024-09-17T04:34:00.143288+0000","last_became_peered":"2024-09-17T04:34:00.143288+0000","last_unstale":"2024-09-17T04:34:01.536532+0000","last_undegraded":"2024-09-17T04:34:01.536532+0000","last_fullsized":"2024-09-17T04:34:01.536532+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:43:32.412802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134818+0000","last_change":"2024-09-17T04:33:59.109283+0000","last_active":"2024-09-17T04:34:01.134818+0000","last_peered":"2024-09-17T04:34:01.134818+0000","last_clean":"2024-09-17T04:34:01.134818+0000","last_became_active":"2024-09-17T04:33:59.108953+0000","last_became_peered":"2024-09-17T04:33:59.108953+0000","last_unstale":"2024-09-17T04:34:01.134818+0000","last_undegraded":"2024-09-17T04:34:01.134818+0000","last_fullsized":"2024-09-17T04:34:01.134818+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:39.786291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542333+0000","last_change":"2024-09-17T04:34:00.138877+0000","last_active":"2024-09-17T04:34:01.542333+0000","last_peered":"2024-09-17T04:34:01.542333+0000","last_clean":"2024-09-17T04:34:01.542333+0000","last_became_active":"2024-09-17T04:34:00.138529+0000","last_became_peered":"2024-09-17T04:34:00.138529+0000","last_unstale":"2024-09-17T04:34:01.542333+0000","last_undegraded":"2024-09-17T04:34:01.542333+0000","last_fullsized":"2024-09-17T04:34:01.542333+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:05:34.986501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134665+0000","last_change":"2024-09-17T04:33:59.112268+0000","last_active":"2024-09-17T04:34:01.134665+0000","last_peered":"2024-09-17T04:34:01.134665+0000","last_clean":"2024-09-17T04:34:01.134665+0000","last_became_active":"2024-09-17T04:33:59.111822+0000","last_became_peered":"2024-09-17T04:33:59.111822+0000","last_unstale":"2024-09-17T04:34:01.134665+0000","last_undegraded":"2024-09-17T04:34:01.134665+0000","last_fullsized":"2024-09-17T04:34:01.134665+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:21:44.164301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134777+0000","last_change":"2024-09-17T04:34:00.131812+0000","last_active":"2024-09-17T04:34:01.134777+0000","last_peered":"2024-09-17T04:34:01.134777+0000","last_clean":"2024-09-17T04:34:01.134777+0000","last_became_active":"2024-09-17T04:34:00.131461+0000","last_became_peered":"2024-09-17T04:34:00.131461+0000","last_unstale":"2024-09-17T04:34:01.134777+0000","last_undegraded":"2024-09-17T04:34:01.134777+0000","last_fullsized":"2024-09-17T04:34:01.134777+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:08:13.529442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134780+0000","last_change":"2024-09-17T04:33:59.115490+0000","last_active":"2024-09-17T04:34:01.134780+0000","last_peered":"2024-09-17T04:34:01.134780+0000","last_clean":"2024-09-17T04:34:01.134780+0000","last_became_active":"2024-09-17T04:33:59.115105+0000","last_became_peered":"2024-09-17T04:33:59.115105+0000","last_unstale":"2024-09-17T04:34:01.134780+0000","last_undegraded":"2024-09-17T04:34:01.134780+0000","last_fullsized":"2024-09-17T04:34:01.134780+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:25.941127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.133761+0000","last_change":"2024-09-17T04:34:00.137771+0000","last_active":"2024-09-17T04:34:01.133761+0000","last_peered":"2024-09-17T04:34:01.133761+0000","last_clean":"2024-09-17T04:34:01.133761+0000","last_became_active":"2024-09-17T04:34:00.137424+0000","last_became_peered":"2024-09-17T04:34:00.137424+0000","last_unstale":"2024-09-17T04:34:01.133761+0000","last_undegraded":"2024-09-17T04:34:01.133761+0000","last_fullsized":"2024-09-17T04:34:01.133761+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:08:40.126582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542241+0000","last_change":"2024-09-17T04:33:59.116366+0000","last_active":"2024-09-17T04:34:01.542241+0000","last_peered":"2024-09-17T04:34:01.542241+0000","last_clean":"2024-09-17T04:34:01.542241+0000","last_became_active":"2024-09-17T04:33:59.116060+0000","last_became_peered":"2024-09-17T04:33:59.116060+0000","last_unstale":"2024-09-17T04:34:01.542241+0000","last_undegraded":"2024-09-17T04:34:01.542241+0000","last_fullsized":"2024-09-17T04:34:01.542241+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:23:51.532043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134654+0000","last_change":"2024-09-17T04:34:00.134391+0000","last_active":"2024-09-17T04:34:01.134654+0000","last_peered":"2024-09-17T04:34:01.134654+0000","last_clean":"2024-09-17T04:34:01.134654+0000","last_became_active":"2024-09-17T04:34:00.134057+0000","last_became_peered":"2024-09-17T04:34:00.134057+0000","last_unstale":"2024-09-17T04:34:01.134654+0000","last_undegraded":"2024-09-17T04:34:01.134654+0000","last_fullsized":"2024-09-17T04:34:01.134654+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:18.435735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.425093+0000","last_change":"2024-09-17T04:33:59.116483+0000","last_active":"2024-09-17T04:34:01.425093+0000","last_peered":"2024-09-17T04:34:01.425093+0000","last_clean":"2024-09-17T04:34:01.425093+0000","last_became_active":"2024-09-17T04:33:59.116144+0000","last_became_peered":"2024-09-17T04:33:59.116144+0000","last_unstale":"2024-09-17T04:34:01.425093+0000","last_undegraded":"2024-09-17T04:34:01.425093+0000","last_fullsized":"2024-09-17T04:34:01.425093+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:40.142964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.133680+0000","last_change":"2024-09-17T04:34:00.133387+0000","last_active":"2024-09-17T04:34:01.133680+0000","last_peered":"2024-09-17T04:34:01.133680+0000","last_clean":"2024-09-17T04:34:01.133680+0000","last_became_active":"2024-09-17T04:34:00.133075+0000","last_became_peered":"2024-09-17T04:34:00.133075+0000","last_unstale":"2024-09-17T04:34:01.133680+0000","last_undegraded":"2024-09-17T04:34:01.133680+0000","last_fullsized":"2024-09-17T04:34:01.133680+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:39.142294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134061+0000","last_change":"2024-09-17T04:33:59.116189+0000","last_active":"2024-09-17T04:34:01.134061+0000","last_peered":"2024-09-17T04:34:01.134061+0000","last_clean":"2024-09-17T04:34:01.134061+0000","last_became_active":"2024-09-17T04:33:59.115627+0000","last_became_peered":"2024-09-17T04:33:59.115627+0000","last_unstale":"2024-09-17T04:34:01.134061+0000","last_undegraded":"2024-09-17T04:34:01.134061+0000","last_fullsized":"2024-09-17T04:34:01.134061+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:31.619034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.541891+0000","last_change":"2024-09-17T04:34:00.134940+0000","last_active":"2024-09-17T04:34:01.541891+0000","last_peered":"2024-09-17T04:34:01.541891+0000","last_clean":"2024-09-17T04:34:01.541891+0000","last_became_active":"2024-09-17T04:34:00.134621+0000","last_became_peered":"2024-09-17T04:34:00.134621+0000","last_unstale":"2024-09-17T04:34:01.541891+0000","last_undegraded":"2024-09-17T04:34:01.541891+0000","last_fullsized":"2024-09-17T04:34:01.541891+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:02.203052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.535676+0000","last_change":"2024-09-17T04:33:59.113577+0000","last_active":"2024-09-17T04:34:01.535676+0000","last_peered":"2024-09-17T04:34:01.535676+0000","last_clean":"2024-09-17T04:34:01.535676+0000","last_became_active":"2024-09-17T04:33:59.113210+0000","last_became_peered":"2024-09-17T04:33:59.113210+0000","last_unstale":"2024-09-17T04:34:01.535676+0000","last_undegraded":"2024-09-17T04:34:01.535676+0000","last_fullsized":"2024-09-17T04:34:01.535676+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:06.764372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134718+0000","last_change":"2024-09-17T04:34:00.139613+0000","last_active":"2024-09-17T04:34:01.134718+0000","last_peered":"2024-09-17T04:34:01.134718+0000","last_clean":"2024-09-17T04:34:01.134718+0000","last_became_active":"2024-09-17T04:34:00.139292+0000","last_became_peered":"2024-09-17T04:34:00.139292+0000","last_unstale":"2024-09-17T04:34:01.134718+0000","last_undegraded":"2024-09-17T04:34:01.134718+0000","last_fullsized":"2024-09-17T04:34:01.134718+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:16:46.909065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.543031+0000","last_change":"2024-09-17T04:33:59.116203+0000","last_active":"2024-09-17T04:34:01.543031+0000","last_peered":"2024-09-17T04:34:01.543031+0000","last_clean":"2024-09-17T04:34:01.543031+0000","last_became_active":"2024-09-17T04:33:59.115805+0000","last_became_peered":"2024-09-17T04:33:59.115805+0000","last_unstale":"2024-09-17T04:34:01.543031+0000","last_undegraded":"2024-09-17T04:34:01.543031+0000","last_fullsized":"2024-09-17T04:34:01.543031+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:23:51.532043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.536523+0000","last_change":"2024-09-17T04:34:00.145064+0000","last_active":"2024-09-17T04:34:01.536523+0000","last_peered":"2024-09-17T04:34:01.536523+0000","last_clean":"2024-09-17T04:34:01.536523+0000","last_became_active":"2024-09-17T04:34:00.144786+0000","last_became_peered":"2024-09-17T04:34:00.144786+0000","last_unstale":"2024-09-17T04:34:01.536523+0000","last_undegraded":"2024-09-17T04:34:01.536523+0000","last_fullsized":"2024-09-17T04:34:01.536523+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:58.534910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:02.130019+0000","last_change":"2024-09-17T04:33:59.117234+0000","last_active":"2024-09-17T04:34:02.130019+0000","last_peered":"2024-09-17T04:34:02.130019+0000","last_clean":"2024-09-17T04:34:02.130019+0000","last_became_active":"2024-09-17T04:33:59.116890+0000","last_became_peered":"2024-09-17T04:33:59.116890+0000","last_unstale":"2024-09-17T04:34:02.130019+0000","last_undegraded":"2024-09-17T04:34:02.130019+0000","last_fullsized":"2024-09-17T04:34:02.130019+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:28.423942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542311+0000","last_change":"2024-09-17T04:34:00.144557+0000","last_active":"2024-09-17T04:34:01.542311+0000","last_peered":"2024-09-17T04:34:01.542311+0000","last_clean":"2024-09-17T04:34:01.542311+0000","last_became_active":"2024-09-17T04:34:00.144230+0000","last_became_peered":"2024-09-17T04:34:00.144230+0000","last_unstale":"2024-09-17T04:34:01.542311+0000","last_undegraded":"2024-09-17T04:34:01.542311+0000","last_fullsized":"2024-09-17T04:34:01.542311+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:46:25.341350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542538+0000","last_change":"2024-09-17T04:33:59.119426+0000","last_active":"2024-09-17T04:34:01.542538+0000","last_peered":"2024-09-17T04:34:01.542538+0000","last_clean":"2024-09-17T04:34:01.542538+0000","last_became_active":"2024-09-17T04:33:59.118894+0000","last_became_peered":"2024-09-17T04:33:59.118894+0000","last_unstale":"2024-09-17T04:34:01.542538+0000","last_undegraded":"2024-09-17T04:34:01.542538+0000","last_fullsized":"2024-09-17T04:34:01.542538+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:31.864330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.425276+0000","last_change":"2024-09-17T04:34:00.135067+0000","last_active":"2024-09-17T04:34:01.425276+0000","last_peered":"2024-09-17T04:34:01.425276+0000","last_clean":"2024-09-17T04:34:01.425276+0000","last_became_active":"2024-09-17T04:34:00.134742+0000","last_became_peered":"2024-09-17T04:34:00.134742+0000","last_unstale":"2024-09-17T04:34:01.425276+0000","last_undegraded":"2024-09-17T04:34:01.425276+0000","last_fullsized":"2024-09-17T04:34:01.425276+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:55.679577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542448+0000","last_change":"2024-09-17T04:33:59.114304+0000","last_active":"2024-09-17T04:34:01.542448+0000","last_peered":"2024-09-17T04:34:01.542448+0000","last_clean":"2024-09-17T04:34:01.542448+0000","last_became_active":"2024-09-17T04:33:59.113930+0000","last_became_peered":"2024-09-17T04:33:59.113930+0000","last_unstale":"2024-09-17T04:34:01.542448+0000","last_undegraded":"2024-09-17T04:34:01.542448+0000","last_fullsized":"2024-09-17T04:34:01.542448+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:11.433911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542543+0000","last_change":"2024-09-17T04:34:00.133293+0000","last_active":"2024-09-17T04:34:01.542543+0000","last_peered":"2024-09-17T04:34:01.542543+0000","last_clean":"2024-09-17T04:34:01.542543+0000","last_became_active":"2024-09-17T04:34:00.132910+0000","last_became_peered":"2024-09-17T04:34:00.132910+0000","last_unstale":"2024-09-17T04:34:01.542543+0000","last_undegraded":"2024-09-17T04:34:01.542543+0000","last_fullsized":"2024-09-17T04:34:01.542543+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:03.455957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542263+0000","last_change":"2024-09-17T04:33:59.119769+0000","last_active":"2024-09-17T04:34:01.542263+0000","last_peered":"2024-09-17T04:34:01.542263+0000","last_clean":"2024-09-17T04:34:01.542263+0000","last_became_active":"2024-09-17T04:33:59.119378+0000","last_became_peered":"2024-09-17T04:33:59.119378+0000","last_unstale":"2024-09-17T04:34:01.542263+0000","last_undegraded":"2024-09-17T04:34:01.542263+0000","last_fullsized":"2024-09-17T04:34:01.542263+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:02.777362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134147+0000","last_change":"2024-09-17T04:34:00.146323+0000","last_active":"2024-09-17T04:34:01.134147+0000","last_peered":"2024-09-17T04:34:01.134147+0000","last_clean":"2024-09-17T04:34:01.134147+0000","last_became_active":"2024-09-17T04:34:00.145997+0000","last_became_peered":"2024-09-17T04:34:00.145997+0000","last_unstale":"2024-09-17T04:34:01.134147+0000","last_undegraded":"2024-09-17T04:34:01.134147+0000","last_fullsized":"2024-09-17T04:34:01.134147+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:41.305573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134180+0000","last_change":"2024-09-17T04:33:59.109854+0000","last_active":"2024-09-17T04:34:01.134180+0000","last_peered":"2024-09-17T04:34:01.134180+0000","last_clean":"2024-09-17T04:34:01.134180+0000","last_became_active":"2024-09-17T04:33:59.109534+0000","last_became_peered":"2024-09-17T04:33:59.109534+0000","last_unstale":"2024-09-17T04:34:01.134180+0000","last_undegraded":"2024-09-17T04:34:01.134180+0000","last_fullsized":"2024-09-17T04:34:01.134180+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:24.369490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134207+0000","last_change":"2024-09-17T04:34:00.146380+0000","last_active":"2024-09-17T04:34:01.134207+0000","last_peered":"2024-09-17T04:34:01.134207+0000","last_clean":"2024-09-17T04:34:01.134207+0000","last_became_active":"2024-09-17T04:34:00.146024+0000","last_became_peered":"2024-09-17T04:34:00.146024+0000","last_unstale":"2024-09-17T04:34:01.134207+0000","last_undegraded":"2024-09-17T04:34:01.134207+0000","last_fullsized":"2024-09-17T04:34:01.134207+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:35:51.784535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134211+0000","last_change":"2024-09-17T04:33:59.121607+0000","last_active":"2024-09-17T04:34:01.134211+0000","last_peered":"2024-09-17T04:34:01.134211+0000","last_clean":"2024-09-17T04:34:01.134211+0000","last_became_active":"2024-09-17T04:33:59.121262+0000","last_became_peered":"2024-09-17T04:33:59.121262+0000","last_unstale":"2024-09-17T04:34:01.134211+0000","last_undegraded":"2024-09-17T04:34:01.134211+0000","last_fullsized":"2024-09-17T04:34:01.134211+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:06.764372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.541922+0000","last_change":"2024-09-17T04:34:00.136520+0000","last_active":"2024-09-17T04:34:01.541922+0000","last_peered":"2024-09-17T04:34:01.541922+0000","last_clean":"2024-09-17T04:34:01.541922+0000","last_became_active":"2024-09-17T04:34:00.136200+0000","last_became_peered":"2024-09-17T04:34:00.136200+0000","last_unstale":"2024-09-17T04:34:01.541922+0000","last_undegraded":"2024-09-17T04:34:01.541922+0000","last_fullsized":"2024-09-17T04:34:01.541922+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:38:06.615802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.424981+0000","last_change":"2024-09-17T04:33:59.116928+0000","last_active":"2024-09-17T04:34:01.424981+0000","last_peered":"2024-09-17T04:34:01.424981+0000","last_clean":"2024-09-17T04:34:01.424981+0000","last_became_active":"2024-09-17T04:33:59.115764+0000","last_became_peered":"2024-09-17T04:33:59.115764+0000","last_unstale":"2024-09-17T04:34:01.424981+0000","last_undegraded":"2024-09-17T04:34:01.424981+0000","last_fullsized":"2024-09-17T04:34:01.424981+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:08.489333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":131,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542243+0000","last_change":"2024-09-17T04:33:09.978742+0000","last_active":"2024-09-17T04:34:01.542243+0000","last_peered":"2024-09-17T04:34:01.542243+0000","last_clean":"2024-09-17T04:34:01.542243+0000","last_became_active":"2024-09-17T04:33:09.977496+0000","last_became_peered":"2024-09-17T04:33:09.977496+0000","last_unstale":"2024-09-17T04:34:01.542243+0000","last_undegraded":"2024-09-17T04:34:01.542243+0000","last_fullsized":"2024-09-17T04:34:01.542243+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:08.930340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:08.930340+0000","last_clean_scrub_stamp":"2024-09-17T04:33:08.930340+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:21:26.566102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542159,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":966656,"data_stored":732558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":376832,"data_stored":142716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542160,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":376832,"data_stored":142772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542159,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":142204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":385024,"data_stored":143734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542159,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":372736,"data_stored":142170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542158,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":385024,"data_stored":143734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542159,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":962560,"data_stored":732468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:34:04.201 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:34:04.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.354+0000 7f8b994c5640 1 -- 172.21.15.37:0/662712275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b940d1a10 msgr2=0x7f8b940ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:04.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.354+0000 7f8b994c5640 1 --2- 172.21.15.37:0/662712275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b940d1a10 0x7f8b940ccb50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8b94073ef0 tx=0x7f8b7c02a560 comp rx=0 tx=0).stop 2024-09-17T04:34:04.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.354+0000 7f8b994c5640 1 -- 172.21.15.37:0/662712275 shutdown_connections 2024-09-17T04:34:04.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.354+0000 7f8b994c5640 1 --2- 172.21.15.37:0/662712275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b940d1a10 0x7f8b940ccb50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.354+0000 7f8b994c5640 1 --2- 172.21.15.37:0/662712275 >> v2:172.21.15.92:3300/0 conn(0x7f8b9415c830 0x7f8b940d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.354+0000 7f8b994c5640 1 -- 172.21.15.37:0/662712275 >> 172.21.15.37:0/662712275 conn(0x7f8b94071050 msgr2=0x7f8b94071460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:04.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.354+0000 7f8b994c5640 1 -- 172.21.15.37:0/662712275 shutdown_connections 2024-09-17T04:34:04.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.354+0000 7f8b994c5640 1 -- 172.21.15.37:0/662712275 wait complete. 2024-09-17T04:34:04.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.356+0000 7f8b994c5640 1 Processor -- start 2024-09-17T04:34:04.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.356+0000 7f8b994c5640 1 -- start start 2024-09-17T04:34:04.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.357+0000 7f8b994c5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b940cd090 0x7f8b94156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:04.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.357+0000 7f8b994c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b940d1a10 0x7f8b941e5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:04.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.357+0000 7f8b994c5640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8b9415c830 0x7f8b940cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:04.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.357+0000 7f8b994c5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8b94072f00 con 0x7f8b940d1a10 2024-09-17T04:34:04.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.357+0000 7f8b994c5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8b940b6920 con 0x7f8b940cd090 2024-09-17T04:34:04.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.357+0000 7f8b994c5640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f8b94073080 con 0x7f8b9415c830 2024-09-17T04:34:04.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.357+0000 7f8b927fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b940d1a10 0x7f8b941e5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:04.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.357+0000 7f8b927fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b940d1a10 0x7f8b941e5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34996/0 (socket says 172.21.15.37:34996) 2024-09-17T04:34:04.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.357+0000 7f8b927fc640 1 -- 172.21.15.37:0/3307868988 learned_addr learned my addr 172.21.15.37:0/3307868988 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:04.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.357+0000 7f8b92ffd640 1 --2- 172.21.15.37:0/3307868988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b940cd090 0x7f8b94156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:04.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.357+0000 7f8b937fe640 1 --2- 172.21.15.37:0/3307868988 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8b9415c830 0x7f8b940cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:04.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.357+0000 7f8b927fc640 1 -- 172.21.15.37:0/3307868988 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8b9415c830 msgr2=0x7f8b940cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:04.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.358+0000 7f8b927fc640 1 --2- 172.21.15.37:0/3307868988 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8b9415c830 0x7f8b940cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.358+0000 7f8b927fc640 1 -- 172.21.15.37:0/3307868988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b940cd090 msgr2=0x7f8b94156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:04.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.358+0000 7f8b927fc640 1 --2- 172.21.15.37:0/3307868988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b940cd090 0x7f8b94156d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.358+0000 7f8b927fc640 1 -- 172.21.15.37:0/3307868988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b941576c0 con 0x7f8b940d1a10 2024-09-17T04:34:04.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.358+0000 7f8b927fc640 1 --2- 172.21.15.37:0/3307868988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b940d1a10 0x7f8b941e5110 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c0099c0 tx=0x7f8b7c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:04.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.358+0000 7f8b8b7fe640 1 -- 172.21.15.37:0/3307868988 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c03e030 con 0x7f8b940d1a10 2024-09-17T04:34:04.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.358+0000 7f8b8b7fe640 1 -- 172.21.15.37:0/3307868988 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b7c0047a0 con 0x7f8b940d1a10 2024-09-17T04:34:04.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.358+0000 7f8b994c5640 1 -- 172.21.15.37:0/3307868988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b941579a0 con 0x7f8b940d1a10 2024-09-17T04:34:04.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.358+0000 7f8b8b7fe640 1 -- 172.21.15.37:0/3307868988 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c035040 con 0x7f8b940d1a10 2024-09-17T04:34:04.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.358+0000 7f8b994c5640 1 -- 172.21.15.37:0/3307868988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b94157f60 con 0x7f8b940d1a10 2024-09-17T04:34:04.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.359+0000 7f8b994c5640 1 -- 172.21.15.37:0/3307868988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b94157430 con 0x7f8b940d1a10 2024-09-17T04:34:04.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.360+0000 7f8b8b7fe640 1 -- 172.21.15.37:0/3307868988 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8b7c041030 con 0x7f8b940d1a10 2024-09-17T04:34:04.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.360+0000 7f8b8b7fe640 1 --2- 172.21.15.37:0/3307868988 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8b74078b90 0x7f8b7407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:04.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.360+0000 7f8b8b7fe640 1 -- 172.21.15.37:0/3307868988 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8b7c0bb5a0 con 0x7f8b940d1a10 2024-09-17T04:34:04.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.360+0000 7f8b92ffd640 1 --2- 172.21.15.37:0/3307868988 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8b74078b90 0x7f8b7407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:04.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.361+0000 7f8b92ffd640 1 --2- 172.21.15.37:0/3307868988 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8b74078b90 0x7f8b7407b0a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8b84005fd0 tx=0x7f8b8400a250 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:04.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.365+0000 7f8b8b7fe640 1 -- 172.21.15.37:0/3307868988 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8b940cd4a0 con 0x7f8b940d1a10 2024-09-17T04:34:04.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.546+0000 7f8b994c5640 1 -- 172.21.15.37:0/3307868988 --> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8b9406a6d0 con 0x7f8b74078b90 2024-09-17T04:34:04.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.551+0000 7f8b8b7fe640 1 -- 172.21.15.37:0/3307868988 <== mgr.4100 v2:172.21.15.92:6800/864337835 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251939 (secure 0 0 0) 0x7f8b9406a6d0 con 0x7f8b74078b90 2024-09-17T04:34:04.553 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:04.554 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-17T04:34:04.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.553+0000 7f8b994c5640 1 -- 172.21.15.37:0/3307868988 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8b74078b90 msgr2=0x7f8b7407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:04.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.553+0000 7f8b994c5640 1 --2- 172.21.15.37:0/3307868988 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8b74078b90 0x7f8b7407b0a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8b84005fd0 tx=0x7f8b8400a250 comp rx=0 tx=0).stop 2024-09-17T04:34:04.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.553+0000 7f8b994c5640 1 -- 172.21.15.37:0/3307868988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b940d1a10 msgr2=0x7f8b941e5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:04.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.553+0000 7f8b994c5640 1 --2- 172.21.15.37:0/3307868988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b940d1a10 0x7f8b941e5110 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c0099c0 tx=0x7f8b7c0039e0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.553+0000 7f8b994c5640 1 -- 172.21.15.37:0/3307868988 shutdown_connections 2024-09-17T04:34:04.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.553+0000 7f8b994c5640 1 --2- 172.21.15.37:0/3307868988 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8b74078b90 0x7f8b7407b0a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.553+0000 7f8b994c5640 1 --2- 172.21.15.37:0/3307868988 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8b9415c830 0x7f8b940cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.553+0000 7f8b994c5640 1 --2- 172.21.15.37:0/3307868988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b940d1a10 0x7f8b941e5110 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.553+0000 7f8b994c5640 1 --2- 172.21.15.37:0/3307868988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b940cd090 0x7f8b94156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.553+0000 7f8b994c5640 1 -- 172.21.15.37:0/3307868988 >> 172.21.15.37:0/3307868988 conn(0x7f8b94071050 msgr2=0x7f8b940b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:04.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.554+0000 7f8b994c5640 1 -- 172.21.15.37:0/3307868988 shutdown_connections 2024-09-17T04:34:04.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.554+0000 7f8b994c5640 1 -- 172.21.15.37:0/3307868988 wait complete. 2024-09-17T04:34:04.565 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-09-17T04:34:04.456526+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":144,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220364,"kb_used_data":4104,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754356,"statfs":{"total":773094113280,"available":772868460544,"internally_reserved":0,"allocated":4202496,"data_stored":2322356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001327"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.425136+0000","last_change":"2024-09-17T04:34:00.133489+0000","last_active":"2024-09-17T04:34:01.425136+0000","last_peered":"2024-09-17T04:34:01.425136+0000","last_clean":"2024-09-17T04:34:01.425136+0000","last_became_active":"2024-09-17T04:34:00.133172+0000","last_became_peered":"2024-09-17T04:34:00.133172+0000","last_unstale":"2024-09-17T04:34:01.425136+0000","last_undegraded":"2024-09-17T04:34:01.425136+0000","last_fullsized":"2024-09-17T04:34:01.425136+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:34.594760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134534+0000","last_change":"2024-09-17T04:34:00.141408+0000","last_active":"2024-09-17T04:34:01.134534+0000","last_peered":"2024-09-17T04:34:01.134534+0000","last_clean":"2024-09-17T04:34:01.134534+0000","last_became_active":"2024-09-17T04:34:00.141077+0000","last_became_peered":"2024-09-17T04:34:00.141077+0000","last_unstale":"2024-09-17T04:34:01.134534+0000","last_undegraded":"2024-09-17T04:34:01.134534+0000","last_fullsized":"2024-09-17T04:34:01.134534+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:24:35.627644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.536143+0000","last_change":"2024-09-17T04:34:00.146084+0000","last_active":"2024-09-17T04:34:01.536143+0000","last_peered":"2024-09-17T04:34:01.536143+0000","last_clean":"2024-09-17T04:34:01.536143+0000","last_became_active":"2024-09-17T04:34:00.145314+0000","last_became_peered":"2024-09-17T04:34:00.145314+0000","last_unstale":"2024-09-17T04:34:01.536143+0000","last_undegraded":"2024-09-17T04:34:01.536143+0000","last_fullsized":"2024-09-17T04:34:01.536143+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:25.874838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542085+0000","last_change":"2024-09-17T04:34:00.138845+0000","last_active":"2024-09-17T04:34:01.542085+0000","last_peered":"2024-09-17T04:34:01.542085+0000","last_clean":"2024-09-17T04:34:01.542085+0000","last_became_active":"2024-09-17T04:34:00.138517+0000","last_became_peered":"2024-09-17T04:34:00.138517+0000","last_unstale":"2024-09-17T04:34:01.542085+0000","last_undegraded":"2024-09-17T04:34:01.542085+0000","last_fullsized":"2024-09-17T04:34:01.542085+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:17.893360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.133673+0000","last_change":"2024-09-17T04:34:00.148227+0000","last_active":"2024-09-17T04:34:01.133673+0000","last_peered":"2024-09-17T04:34:01.133673+0000","last_clean":"2024-09-17T04:34:01.133673+0000","last_became_active":"2024-09-17T04:34:00.147902+0000","last_became_peered":"2024-09-17T04:34:00.147902+0000","last_unstale":"2024-09-17T04:34:01.133673+0000","last_undegraded":"2024-09-17T04:34:01.133673+0000","last_fullsized":"2024-09-17T04:34:01.133673+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:52.789669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134277+0000","last_change":"2024-09-17T04:34:00.139638+0000","last_active":"2024-09-17T04:34:01.134277+0000","last_peered":"2024-09-17T04:34:01.134277+0000","last_clean":"2024-09-17T04:34:01.134277+0000","last_became_active":"2024-09-17T04:34:00.139252+0000","last_became_peered":"2024-09-17T04:34:00.139252+0000","last_unstale":"2024-09-17T04:34:01.134277+0000","last_undegraded":"2024-09-17T04:34:01.134277+0000","last_fullsized":"2024-09-17T04:34:01.134277+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:42.085058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542050+0000","last_change":"2024-09-17T04:34:00.135963+0000","last_active":"2024-09-17T04:34:01.542050+0000","last_peered":"2024-09-17T04:34:01.542050+0000","last_clean":"2024-09-17T04:34:01.542050+0000","last_became_active":"2024-09-17T04:34:00.135466+0000","last_became_peered":"2024-09-17T04:34:00.135466+0000","last_unstale":"2024-09-17T04:34:01.542050+0000","last_undegraded":"2024-09-17T04:34:01.542050+0000","last_fullsized":"2024-09-17T04:34:01.542050+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:49:15.290254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.541777+0000","last_change":"2024-09-17T04:34:00.136461+0000","last_active":"2024-09-17T04:34:01.541777+0000","last_peered":"2024-09-17T04:34:01.541777+0000","last_clean":"2024-09-17T04:34:01.541777+0000","last_became_active":"2024-09-17T04:34:00.136127+0000","last_became_peered":"2024-09-17T04:34:00.136127+0000","last_unstale":"2024-09-17T04:34:01.541777+0000","last_undegraded":"2024-09-17T04:34:01.541777+0000","last_fullsized":"2024-09-17T04:34:01.541777+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:57:08.760910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.135030+0000","last_change":"2024-09-17T04:34:00.126244+0000","last_active":"2024-09-17T04:34:01.135030+0000","last_peered":"2024-09-17T04:34:01.135030+0000","last_clean":"2024-09-17T04:34:01.135030+0000","last_became_active":"2024-09-17T04:34:00.125921+0000","last_became_peered":"2024-09-17T04:34:00.125921+0000","last_unstale":"2024-09-17T04:34:01.135030+0000","last_undegraded":"2024-09-17T04:34:01.135030+0000","last_fullsized":"2024-09-17T04:34:01.135030+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:21:47.588781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542872+0000","last_change":"2024-09-17T04:34:00.147614+0000","last_active":"2024-09-17T04:34:01.542872+0000","last_peered":"2024-09-17T04:34:01.542872+0000","last_clean":"2024-09-17T04:34:01.542872+0000","last_became_active":"2024-09-17T04:34:00.147198+0000","last_became_peered":"2024-09-17T04:34:00.147198+0000","last_unstale":"2024-09-17T04:34:01.542872+0000","last_undegraded":"2024-09-17T04:34:01.542872+0000","last_fullsized":"2024-09-17T04:34:01.542872+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:02.203052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542113+0000","last_change":"2024-09-17T04:34:00.144759+0000","last_active":"2024-09-17T04:34:01.542113+0000","last_peered":"2024-09-17T04:34:01.542113+0000","last_clean":"2024-09-17T04:34:01.542113+0000","last_became_active":"2024-09-17T04:34:00.144463+0000","last_became_peered":"2024-09-17T04:34:00.144463+0000","last_unstale":"2024-09-17T04:34:01.542113+0000","last_undegraded":"2024-09-17T04:34:01.542113+0000","last_fullsized":"2024-09-17T04:34:01.542113+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:09:46.692534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.536185+0000","last_change":"2024-09-17T04:34:00.144495+0000","last_active":"2024-09-17T04:34:01.536185+0000","last_peered":"2024-09-17T04:34:01.536185+0000","last_clean":"2024-09-17T04:34:01.536185+0000","last_became_active":"2024-09-17T04:34:00.144027+0000","last_became_peered":"2024-09-17T04:34:00.144027+0000","last_unstale":"2024-09-17T04:34:01.536185+0000","last_undegraded":"2024-09-17T04:34:01.536185+0000","last_fullsized":"2024-09-17T04:34:01.536185+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:09:46.692534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.133926+0000","last_change":"2024-09-17T04:34:00.137663+0000","last_active":"2024-09-17T04:34:01.133926+0000","last_peered":"2024-09-17T04:34:01.133926+0000","last_clean":"2024-09-17T04:34:01.133926+0000","last_became_active":"2024-09-17T04:34:00.137306+0000","last_became_peered":"2024-09-17T04:34:00.137306+0000","last_unstale":"2024-09-17T04:34:01.133926+0000","last_undegraded":"2024-09-17T04:34:01.133926+0000","last_fullsized":"2024-09-17T04:34:01.133926+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:24.590166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134622+0000","last_change":"2024-09-17T04:34:00.140479+0000","last_active":"2024-09-17T04:34:01.134622+0000","last_peered":"2024-09-17T04:34:01.134622+0000","last_clean":"2024-09-17T04:34:01.134622+0000","last_became_active":"2024-09-17T04:34:00.140125+0000","last_became_peered":"2024-09-17T04:34:00.140125+0000","last_unstale":"2024-09-17T04:34:01.134622+0000","last_undegraded":"2024-09-17T04:34:01.134622+0000","last_fullsized":"2024-09-17T04:34:01.134622+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:05.573033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.535981+0000","last_change":"2024-09-17T04:34:00.135929+0000","last_active":"2024-09-17T04:34:01.535981+0000","last_peered":"2024-09-17T04:34:01.535981+0000","last_clean":"2024-09-17T04:34:01.535981+0000","last_became_active":"2024-09-17T04:34:00.135615+0000","last_became_peered":"2024-09-17T04:34:00.135615+0000","last_unstale":"2024-09-17T04:34:01.535981+0000","last_undegraded":"2024-09-17T04:34:01.535981+0000","last_fullsized":"2024-09-17T04:34:01.535981+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:47:51.639001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542164+0000","last_change":"2024-09-17T04:34:00.147088+0000","last_active":"2024-09-17T04:34:01.542164+0000","last_peered":"2024-09-17T04:34:01.542164+0000","last_clean":"2024-09-17T04:34:01.542164+0000","last_became_active":"2024-09-17T04:34:00.146778+0000","last_became_peered":"2024-09-17T04:34:00.146778+0000","last_unstale":"2024-09-17T04:34:01.542164+0000","last_undegraded":"2024-09-17T04:34:01.542164+0000","last_fullsized":"2024-09-17T04:34:01.542164+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:38.015410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542362+0000","last_change":"2024-09-17T04:34:00.134671+0000","last_active":"2024-09-17T04:34:01.542362+0000","last_peered":"2024-09-17T04:34:01.542362+0000","last_clean":"2024-09-17T04:34:01.542362+0000","last_became_active":"2024-09-17T04:34:00.132323+0000","last_became_peered":"2024-09-17T04:34:00.132323+0000","last_unstale":"2024-09-17T04:34:01.542362+0000","last_undegraded":"2024-09-17T04:34:01.542362+0000","last_fullsized":"2024-09-17T04:34:01.542362+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:58.534910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134576+0000","last_change":"2024-09-17T04:34:00.147773+0000","last_active":"2024-09-17T04:34:01.134576+0000","last_peered":"2024-09-17T04:34:01.134576+0000","last_clean":"2024-09-17T04:34:01.134576+0000","last_became_active":"2024-09-17T04:34:00.147077+0000","last_became_peered":"2024-09-17T04:34:00.147077+0000","last_unstale":"2024-09-17T04:34:01.134576+0000","last_undegraded":"2024-09-17T04:34:01.134576+0000","last_fullsized":"2024-09-17T04:34:01.134576+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:04:29.231102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.133935+0000","last_change":"2024-09-17T04:34:00.149546+0000","last_active":"2024-09-17T04:34:01.133935+0000","last_peered":"2024-09-17T04:34:01.133935+0000","last_clean":"2024-09-17T04:34:01.133935+0000","last_became_active":"2024-09-17T04:34:00.149166+0000","last_became_peered":"2024-09-17T04:34:00.149166+0000","last_unstale":"2024-09-17T04:34:01.133935+0000","last_undegraded":"2024-09-17T04:34:01.133935+0000","last_fullsized":"2024-09-17T04:34:01.133935+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:09:46.692534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134934+0000","last_change":"2024-09-17T04:34:00.140053+0000","last_active":"2024-09-17T04:34:01.134934+0000","last_peered":"2024-09-17T04:34:01.134934+0000","last_clean":"2024-09-17T04:34:01.134934+0000","last_became_active":"2024-09-17T04:34:00.139557+0000","last_became_peered":"2024-09-17T04:34:00.139557+0000","last_unstale":"2024-09-17T04:34:01.134934+0000","last_undegraded":"2024-09-17T04:34:01.134934+0000","last_fullsized":"2024-09-17T04:34:01.134934+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:50:59.831340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134657+0000","last_change":"2024-09-17T04:34:00.133295+0000","last_active":"2024-09-17T04:34:01.134657+0000","last_peered":"2024-09-17T04:34:01.134657+0000","last_clean":"2024-09-17T04:34:01.134657+0000","last_became_active":"2024-09-17T04:34:00.132965+0000","last_became_peered":"2024-09-17T04:34:00.132965+0000","last_unstale":"2024-09-17T04:34:01.134657+0000","last_undegraded":"2024-09-17T04:34:01.134657+0000","last_fullsized":"2024-09-17T04:34:01.134657+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:52.024315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134610+0000","last_change":"2024-09-17T04:34:00.148488+0000","last_active":"2024-09-17T04:34:01.134610+0000","last_peered":"2024-09-17T04:34:01.134610+0000","last_clean":"2024-09-17T04:34:01.134610+0000","last_became_active":"2024-09-17T04:34:00.148176+0000","last_became_peered":"2024-09-17T04:34:00.148176+0000","last_unstale":"2024-09-17T04:34:01.134610+0000","last_undegraded":"2024-09-17T04:34:01.134610+0000","last_fullsized":"2024-09-17T04:34:01.134610+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:23.552526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.541908+0000","last_change":"2024-09-17T04:34:00.132040+0000","last_active":"2024-09-17T04:34:01.541908+0000","last_peered":"2024-09-17T04:34:01.541908+0000","last_clean":"2024-09-17T04:34:01.541908+0000","last_became_active":"2024-09-17T04:34:00.131717+0000","last_became_peered":"2024-09-17T04:34:00.131717+0000","last_unstale":"2024-09-17T04:34:01.541908+0000","last_undegraded":"2024-09-17T04:34:01.541908+0000","last_fullsized":"2024-09-17T04:34:01.541908+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:27.558398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.535779+0000","last_change":"2024-09-17T04:34:00.137953+0000","last_active":"2024-09-17T04:34:01.535779+0000","last_peered":"2024-09-17T04:34:01.535779+0000","last_clean":"2024-09-17T04:34:01.535779+0000","last_became_active":"2024-09-17T04:34:00.137577+0000","last_became_peered":"2024-09-17T04:34:00.137577+0000","last_unstale":"2024-09-17T04:34:01.535779+0000","last_undegraded":"2024-09-17T04:34:01.535779+0000","last_fullsized":"2024-09-17T04:34:01.535779+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:33:39.300267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542955+0000","last_change":"2024-09-17T04:34:00.134911+0000","last_active":"2024-09-17T04:34:01.542955+0000","last_peered":"2024-09-17T04:34:01.542955+0000","last_clean":"2024-09-17T04:34:01.542955+0000","last_became_active":"2024-09-17T04:34:00.133373+0000","last_became_peered":"2024-09-17T04:34:00.133373+0000","last_unstale":"2024-09-17T04:34:01.542955+0000","last_undegraded":"2024-09-17T04:34:01.542955+0000","last_fullsized":"2024-09-17T04:34:01.542955+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:27:05.531961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134763+0000","last_change":"2024-09-17T04:34:00.147296+0000","last_active":"2024-09-17T04:34:01.134763+0000","last_peered":"2024-09-17T04:34:01.134763+0000","last_clean":"2024-09-17T04:34:01.134763+0000","last_became_active":"2024-09-17T04:34:00.146984+0000","last_became_peered":"2024-09-17T04:34:00.146984+0000","last_unstale":"2024-09-17T04:34:01.134763+0000","last_undegraded":"2024-09-17T04:34:01.134763+0000","last_fullsized":"2024-09-17T04:34:01.134763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:51.136426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542793+0000","last_change":"2024-09-17T04:34:00.136351+0000","last_active":"2024-09-17T04:34:01.542793+0000","last_peered":"2024-09-17T04:34:01.542793+0000","last_clean":"2024-09-17T04:34:01.542793+0000","last_became_active":"2024-09-17T04:34:00.136007+0000","last_became_peered":"2024-09-17T04:34:00.136007+0000","last_unstale":"2024-09-17T04:34:01.542793+0000","last_undegraded":"2024-09-17T04:34:01.542793+0000","last_fullsized":"2024-09-17T04:34:01.542793+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:37:23.029519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.536329+0000","last_change":"2024-09-17T04:34:00.134272+0000","last_active":"2024-09-17T04:34:01.536329+0000","last_peered":"2024-09-17T04:34:01.536329+0000","last_clean":"2024-09-17T04:34:01.536329+0000","last_became_active":"2024-09-17T04:34:00.133409+0000","last_became_peered":"2024-09-17T04:34:00.133409+0000","last_unstale":"2024-09-17T04:34:01.536329+0000","last_undegraded":"2024-09-17T04:34:01.536329+0000","last_fullsized":"2024-09-17T04:34:01.536329+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:38.015410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134538+0000","last_change":"2024-09-17T04:34:00.146595+0000","last_active":"2024-09-17T04:34:01.134538+0000","last_peered":"2024-09-17T04:34:01.134538+0000","last_clean":"2024-09-17T04:34:01.134538+0000","last_became_active":"2024-09-17T04:34:00.145812+0000","last_became_peered":"2024-09-17T04:34:00.145812+0000","last_unstale":"2024-09-17T04:34:01.134538+0000","last_undegraded":"2024-09-17T04:34:01.134538+0000","last_fullsized":"2024-09-17T04:34:01.134538+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:15.949787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.541869+0000","last_change":"2024-09-17T04:34:00.131809+0000","last_active":"2024-09-17T04:34:01.541869+0000","last_peered":"2024-09-17T04:34:01.541869+0000","last_clean":"2024-09-17T04:34:01.541869+0000","last_became_active":"2024-09-17T04:34:00.131532+0000","last_became_peered":"2024-09-17T04:34:00.131532+0000","last_unstale":"2024-09-17T04:34:01.541869+0000","last_undegraded":"2024-09-17T04:34:01.541869+0000","last_fullsized":"2024-09-17T04:34:01.541869+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:25:50.372117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.425144+0000","last_change":"2024-09-17T04:34:00.140013+0000","last_active":"2024-09-17T04:34:01.425144+0000","last_peered":"2024-09-17T04:34:01.425144+0000","last_clean":"2024-09-17T04:34:01.425144+0000","last_became_active":"2024-09-17T04:34:00.139703+0000","last_became_peered":"2024-09-17T04:34:00.139703+0000","last_unstale":"2024-09-17T04:34:01.425144+0000","last_undegraded":"2024-09-17T04:34:01.425144+0000","last_fullsized":"2024-09-17T04:34:01.425144+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:59:37.912219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.541803+0000","last_change":"2024-09-17T04:34:00.148182+0000","last_active":"2024-09-17T04:34:01.541803+0000","last_peered":"2024-09-17T04:34:01.541803+0000","last_clean":"2024-09-17T04:34:01.541803+0000","last_became_active":"2024-09-17T04:34:00.147537+0000","last_became_peered":"2024-09-17T04:34:00.147537+0000","last_unstale":"2024-09-17T04:34:01.541803+0000","last_undegraded":"2024-09-17T04:34:01.541803+0000","last_fullsized":"2024-09-17T04:34:01.541803+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:37:23.029519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542456+0000","last_change":"2024-09-17T04:34:00.139991+0000","last_active":"2024-09-17T04:34:01.542456+0000","last_peered":"2024-09-17T04:34:01.542456+0000","last_clean":"2024-09-17T04:34:01.542456+0000","last_became_active":"2024-09-17T04:34:00.139640+0000","last_became_peered":"2024-09-17T04:34:00.139640+0000","last_unstale":"2024-09-17T04:34:01.542456+0000","last_undegraded":"2024-09-17T04:34:01.542456+0000","last_fullsized":"2024-09-17T04:34:01.542456+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:57:02.382202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.425229+0000","last_change":"2024-09-17T04:33:59.113533+0000","last_active":"2024-09-17T04:34:01.425229+0000","last_peered":"2024-09-17T04:34:01.425229+0000","last_clean":"2024-09-17T04:34:01.425229+0000","last_became_active":"2024-09-17T04:33:59.113062+0000","last_became_peered":"2024-09-17T04:33:59.113062+0000","last_unstale":"2024-09-17T04:34:01.425229+0000","last_undegraded":"2024-09-17T04:34:01.425229+0000","last_fullsized":"2024-09-17T04:34:01.425229+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:00.673571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.425212+0000","last_change":"2024-09-17T04:34:00.143656+0000","last_active":"2024-09-17T04:34:01.425212+0000","last_peered":"2024-09-17T04:34:01.425212+0000","last_clean":"2024-09-17T04:34:01.425212+0000","last_became_active":"2024-09-17T04:34:00.143327+0000","last_became_peered":"2024-09-17T04:34:00.143327+0000","last_unstale":"2024-09-17T04:34:01.425212+0000","last_undegraded":"2024-09-17T04:34:01.425212+0000","last_fullsized":"2024-09-17T04:34:01.425212+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:02:15.227761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542587+0000","last_change":"2024-09-17T04:33:59.113364+0000","last_active":"2024-09-17T04:34:01.542587+0000","last_peered":"2024-09-17T04:34:01.542587+0000","last_clean":"2024-09-17T04:34:01.542587+0000","last_became_active":"2024-09-17T04:33:59.112021+0000","last_became_peered":"2024-09-17T04:33:59.112021+0000","last_unstale":"2024-09-17T04:34:01.542587+0000","last_undegraded":"2024-09-17T04:34:01.542587+0000","last_fullsized":"2024-09-17T04:34:01.542587+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:53:00.824171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134850+0000","last_change":"2024-09-17T04:34:00.148658+0000","last_active":"2024-09-17T04:34:01.134850+0000","last_peered":"2024-09-17T04:34:01.134850+0000","last_clean":"2024-09-17T04:34:01.134850+0000","last_became_active":"2024-09-17T04:34:00.148350+0000","last_became_peered":"2024-09-17T04:34:00.148350+0000","last_unstale":"2024-09-17T04:34:01.134850+0000","last_undegraded":"2024-09-17T04:34:01.134850+0000","last_fullsized":"2024-09-17T04:34:01.134850+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:25:19.913461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134108+0000","last_change":"2024-09-17T04:33:59.124080+0000","last_active":"2024-09-17T04:34:01.134108+0000","last_peered":"2024-09-17T04:34:01.134108+0000","last_clean":"2024-09-17T04:34:01.134108+0000","last_became_active":"2024-09-17T04:33:59.123624+0000","last_became_peered":"2024-09-17T04:33:59.123624+0000","last_unstale":"2024-09-17T04:34:01.134108+0000","last_undegraded":"2024-09-17T04:34:01.134108+0000","last_fullsized":"2024-09-17T04:34:01.134108+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:52.054448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542775+0000","last_change":"2024-09-17T04:34:00.136203+0000","last_active":"2024-09-17T04:34:01.542775+0000","last_peered":"2024-09-17T04:34:01.542775+0000","last_clean":"2024-09-17T04:34:01.542775+0000","last_became_active":"2024-09-17T04:34:00.135871+0000","last_became_peered":"2024-09-17T04:34:00.135871+0000","last_unstale":"2024-09-17T04:34:01.542775+0000","last_undegraded":"2024-09-17T04:34:01.542775+0000","last_fullsized":"2024-09-17T04:34:01.542775+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:41.010278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134892+0000","last_change":"2024-09-17T04:33:59.119452+0000","last_active":"2024-09-17T04:34:01.134892+0000","last_peered":"2024-09-17T04:34:01.134892+0000","last_clean":"2024-09-17T04:34:01.134892+0000","last_became_active":"2024-09-17T04:33:59.119108+0000","last_became_peered":"2024-09-17T04:33:59.119108+0000","last_unstale":"2024-09-17T04:34:01.134892+0000","last_undegraded":"2024-09-17T04:34:01.134892+0000","last_fullsized":"2024-09-17T04:34:01.134892+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:52.325898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.541792+0000","last_change":"2024-09-17T04:34:00.143488+0000","last_active":"2024-09-17T04:34:01.541792+0000","last_peered":"2024-09-17T04:34:01.541792+0000","last_clean":"2024-09-17T04:34:01.541792+0000","last_became_active":"2024-09-17T04:34:00.143166+0000","last_became_peered":"2024-09-17T04:34:00.143166+0000","last_unstale":"2024-09-17T04:34:01.541792+0000","last_undegraded":"2024-09-17T04:34:01.541792+0000","last_fullsized":"2024-09-17T04:34:01.541792+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:27:05.531961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134677+0000","last_change":"2024-09-17T04:33:59.114365+0000","last_active":"2024-09-17T04:34:01.134677+0000","last_peered":"2024-09-17T04:34:01.134677+0000","last_clean":"2024-09-17T04:34:01.134677+0000","last_became_active":"2024-09-17T04:33:59.114058+0000","last_became_peered":"2024-09-17T04:33:59.114058+0000","last_unstale":"2024-09-17T04:34:01.134677+0000","last_undegraded":"2024-09-17T04:34:01.134677+0000","last_fullsized":"2024-09-17T04:34:01.134677+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:43:41.290086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.536148+0000","last_change":"2024-09-17T04:34:00.142714+0000","last_active":"2024-09-17T04:34:01.536148+0000","last_peered":"2024-09-17T04:34:01.536148+0000","last_clean":"2024-09-17T04:34:01.536148+0000","last_became_active":"2024-09-17T04:34:00.142287+0000","last_became_peered":"2024-09-17T04:34:00.142287+0000","last_unstale":"2024-09-17T04:34:01.536148+0000","last_undegraded":"2024-09-17T04:34:01.536148+0000","last_fullsized":"2024-09-17T04:34:01.536148+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:49:15.290254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134401+0000","last_change":"2024-09-17T04:33:59.117052+0000","last_active":"2024-09-17T04:34:01.134401+0000","last_peered":"2024-09-17T04:34:01.134401+0000","last_clean":"2024-09-17T04:34:01.134401+0000","last_became_active":"2024-09-17T04:33:59.116738+0000","last_became_peered":"2024-09-17T04:33:59.116738+0000","last_unstale":"2024-09-17T04:34:01.134401+0000","last_undegraded":"2024-09-17T04:34:01.134401+0000","last_fullsized":"2024-09-17T04:34:01.134401+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:09.152247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134181+0000","last_change":"2024-09-17T04:34:00.132375+0000","last_active":"2024-09-17T04:34:01.134181+0000","last_peered":"2024-09-17T04:34:01.134181+0000","last_clean":"2024-09-17T04:34:01.134181+0000","last_became_active":"2024-09-17T04:34:00.131490+0000","last_became_peered":"2024-09-17T04:34:00.131490+0000","last_unstale":"2024-09-17T04:34:01.134181+0000","last_undegraded":"2024-09-17T04:34:01.134181+0000","last_fullsized":"2024-09-17T04:34:01.134181+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:56.019994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134132+0000","last_change":"2024-09-17T04:33:59.121968+0000","last_active":"2024-09-17T04:34:01.134132+0000","last_peered":"2024-09-17T04:34:01.134132+0000","last_clean":"2024-09-17T04:34:01.134132+0000","last_became_active":"2024-09-17T04:33:59.121152+0000","last_became_peered":"2024-09-17T04:33:59.121152+0000","last_unstale":"2024-09-17T04:34:01.134132+0000","last_undegraded":"2024-09-17T04:34:01.134132+0000","last_fullsized":"2024-09-17T04:34:01.134132+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:02:14.225662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.541965+0000","last_change":"2024-09-17T04:34:00.137017+0000","last_active":"2024-09-17T04:34:01.541965+0000","last_peered":"2024-09-17T04:34:01.541965+0000","last_clean":"2024-09-17T04:34:01.541965+0000","last_became_active":"2024-09-17T04:34:00.136469+0000","last_became_peered":"2024-09-17T04:34:00.136469+0000","last_unstale":"2024-09-17T04:34:01.541965+0000","last_undegraded":"2024-09-17T04:34:01.541965+0000","last_fullsized":"2024-09-17T04:34:01.541965+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:39:50.199228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134056+0000","last_change":"2024-09-17T04:33:59.109225+0000","last_active":"2024-09-17T04:34:01.134056+0000","last_peered":"2024-09-17T04:34:01.134056+0000","last_clean":"2024-09-17T04:34:01.134056+0000","last_became_active":"2024-09-17T04:33:59.108890+0000","last_became_peered":"2024-09-17T04:33:59.108890+0000","last_unstale":"2024-09-17T04:34:01.134056+0000","last_undegraded":"2024-09-17T04:34:01.134056+0000","last_fullsized":"2024-09-17T04:34:01.134056+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:59.447364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.536396+0000","last_change":"2024-09-17T04:34:00.149518+0000","last_active":"2024-09-17T04:34:01.536396+0000","last_peered":"2024-09-17T04:34:01.536396+0000","last_clean":"2024-09-17T04:34:01.536396+0000","last_became_active":"2024-09-17T04:34:00.149133+0000","last_became_peered":"2024-09-17T04:34:00.149133+0000","last_unstale":"2024-09-17T04:34:01.536396+0000","last_undegraded":"2024-09-17T04:34:01.536396+0000","last_fullsized":"2024-09-17T04:34:01.536396+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:09:37.721140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134378+0000","last_change":"2024-09-17T04:33:59.120720+0000","last_active":"2024-09-17T04:34:01.134378+0000","last_peered":"2024-09-17T04:34:01.134378+0000","last_clean":"2024-09-17T04:34:01.134378+0000","last_became_active":"2024-09-17T04:33:59.120386+0000","last_became_peered":"2024-09-17T04:33:59.120386+0000","last_unstale":"2024-09-17T04:34:01.134378+0000","last_undegraded":"2024-09-17T04:34:01.134378+0000","last_fullsized":"2024-09-17T04:34:01.134378+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:43.682967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134069+0000","last_change":"2024-09-17T04:34:00.150547+0000","last_active":"2024-09-17T04:34:01.134069+0000","last_peered":"2024-09-17T04:34:01.134069+0000","last_clean":"2024-09-17T04:34:01.134069+0000","last_became_active":"2024-09-17T04:34:00.146052+0000","last_became_peered":"2024-09-17T04:34:00.146052+0000","last_unstale":"2024-09-17T04:34:01.134069+0000","last_undegraded":"2024-09-17T04:34:01.134069+0000","last_fullsized":"2024-09-17T04:34:01.134069+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:22:12.941844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542591+0000","last_change":"2024-09-17T04:33:59.112992+0000","last_active":"2024-09-17T04:34:01.542591+0000","last_peered":"2024-09-17T04:34:01.542591+0000","last_clean":"2024-09-17T04:34:01.542591+0000","last_became_active":"2024-09-17T04:33:59.112657+0000","last_became_peered":"2024-09-17T04:33:59.112657+0000","last_unstale":"2024-09-17T04:34:01.542591+0000","last_undegraded":"2024-09-17T04:34:01.542591+0000","last_fullsized":"2024-09-17T04:34:01.542591+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:06.203780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134425+0000","last_change":"2024-09-17T04:34:00.148749+0000","last_active":"2024-09-17T04:34:01.134425+0000","last_peered":"2024-09-17T04:34:01.134425+0000","last_clean":"2024-09-17T04:34:01.134425+0000","last_became_active":"2024-09-17T04:34:00.147951+0000","last_became_peered":"2024-09-17T04:34:00.147951+0000","last_unstale":"2024-09-17T04:34:01.134425+0000","last_undegraded":"2024-09-17T04:34:01.134425+0000","last_fullsized":"2024-09-17T04:34:01.134425+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:46.711607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.536289+0000","last_change":"2024-09-17T04:33:59.112913+0000","last_active":"2024-09-17T04:34:01.536289+0000","last_peered":"2024-09-17T04:34:01.536289+0000","last_clean":"2024-09-17T04:34:01.536289+0000","last_became_active":"2024-09-17T04:33:59.112540+0000","last_became_peered":"2024-09-17T04:33:59.112540+0000","last_unstale":"2024-09-17T04:34:01.536289+0000","last_undegraded":"2024-09-17T04:34:01.536289+0000","last_fullsized":"2024-09-17T04:34:01.536289+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:33.592661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.536113+0000","last_change":"2024-09-17T04:34:00.146174+0000","last_active":"2024-09-17T04:34:01.536113+0000","last_peered":"2024-09-17T04:34:01.536113+0000","last_clean":"2024-09-17T04:34:01.536113+0000","last_became_active":"2024-09-17T04:34:00.145752+0000","last_became_peered":"2024-09-17T04:34:00.145752+0000","last_unstale":"2024-09-17T04:34:01.536113+0000","last_undegraded":"2024-09-17T04:34:01.536113+0000","last_fullsized":"2024-09-17T04:34:01.536113+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:44.352160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134384+0000","last_change":"2024-09-17T04:33:59.124191+0000","last_active":"2024-09-17T04:34:01.134384+0000","last_peered":"2024-09-17T04:34:01.134384+0000","last_clean":"2024-09-17T04:34:01.134384+0000","last_became_active":"2024-09-17T04:33:59.123823+0000","last_became_peered":"2024-09-17T04:33:59.123823+0000","last_unstale":"2024-09-17T04:34:01.134384+0000","last_undegraded":"2024-09-17T04:34:01.134384+0000","last_fullsized":"2024-09-17T04:34:01.134384+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:39:49.197130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542212+0000","last_change":"2024-09-17T04:34:00.132724+0000","last_active":"2024-09-17T04:34:01.542212+0000","last_peered":"2024-09-17T04:34:01.542212+0000","last_clean":"2024-09-17T04:34:01.542212+0000","last_became_active":"2024-09-17T04:34:00.131882+0000","last_became_peered":"2024-09-17T04:34:00.131882+0000","last_unstale":"2024-09-17T04:34:01.542212+0000","last_undegraded":"2024-09-17T04:34:01.542212+0000","last_fullsized":"2024-09-17T04:34:01.542212+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:48.008056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134860+0000","last_change":"2024-09-17T04:33:59.109143+0000","last_active":"2024-09-17T04:34:01.134860+0000","last_peered":"2024-09-17T04:34:01.134860+0000","last_clean":"2024-09-17T04:34:01.134860+0000","last_became_active":"2024-09-17T04:33:59.108847+0000","last_became_peered":"2024-09-17T04:33:59.108847+0000","last_unstale":"2024-09-17T04:34:01.134860+0000","last_undegraded":"2024-09-17T04:34:01.134860+0000","last_fullsized":"2024-09-17T04:34:01.134860+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:18.019734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.133940+0000","last_change":"2024-09-17T04:34:00.145327+0000","last_active":"2024-09-17T04:34:01.133940+0000","last_peered":"2024-09-17T04:34:01.133940+0000","last_clean":"2024-09-17T04:34:01.133940+0000","last_became_active":"2024-09-17T04:34:00.142809+0000","last_became_peered":"2024-09-17T04:34:00.142809+0000","last_unstale":"2024-09-17T04:34:01.133940+0000","last_undegraded":"2024-09-17T04:34:01.133940+0000","last_fullsized":"2024-09-17T04:34:01.133940+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:47:41.427853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542157+0000","last_change":"2024-09-17T04:33:59.116224+0000","last_active":"2024-09-17T04:34:01.542157+0000","last_peered":"2024-09-17T04:34:01.542157+0000","last_clean":"2024-09-17T04:34:01.542157+0000","last_became_active":"2024-09-17T04:33:59.115862+0000","last_became_peered":"2024-09-17T04:33:59.115862+0000","last_unstale":"2024-09-17T04:34:01.542157+0000","last_undegraded":"2024-09-17T04:34:01.542157+0000","last_fullsized":"2024-09-17T04:34:01.542157+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:22:13.092980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542180+0000","last_change":"2024-09-17T04:34:00.148329+0000","last_active":"2024-09-17T04:34:01.542180+0000","last_peered":"2024-09-17T04:34:01.542180+0000","last_clean":"2024-09-17T04:34:01.542180+0000","last_became_active":"2024-09-17T04:34:00.147988+0000","last_became_peered":"2024-09-17T04:34:00.147988+0000","last_unstale":"2024-09-17T04:34:01.542180+0000","last_undegraded":"2024-09-17T04:34:01.542180+0000","last_fullsized":"2024-09-17T04:34:01.542180+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:09:37.721140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134261+0000","last_change":"2024-09-17T04:33:59.120796+0000","last_active":"2024-09-17T04:34:01.134261+0000","last_peered":"2024-09-17T04:34:01.134261+0000","last_clean":"2024-09-17T04:34:01.134261+0000","last_became_active":"2024-09-17T04:33:59.120478+0000","last_became_peered":"2024-09-17T04:33:59.120478+0000","last_unstale":"2024-09-17T04:34:01.134261+0000","last_undegraded":"2024-09-17T04:34:01.134261+0000","last_fullsized":"2024-09-17T04:34:01.134261+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:54.677479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.425130+0000","last_change":"2024-09-17T04:34:00.147713+0000","last_active":"2024-09-17T04:34:01.425130+0000","last_peered":"2024-09-17T04:34:01.425130+0000","last_clean":"2024-09-17T04:34:01.425130+0000","last_became_active":"2024-09-17T04:34:00.147099+0000","last_became_peered":"2024-09-17T04:34:00.147099+0000","last_unstale":"2024-09-17T04:34:01.425130+0000","last_undegraded":"2024-09-17T04:34:01.425130+0000","last_fullsized":"2024-09-17T04:34:01.425130+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:22:45.296651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134091+0000","last_change":"2024-09-17T04:33:59.122217+0000","last_active":"2024-09-17T04:34:01.134091+0000","last_peered":"2024-09-17T04:34:01.134091+0000","last_clean":"2024-09-17T04:34:01.134091+0000","last_became_active":"2024-09-17T04:33:59.121866+0000","last_became_peered":"2024-09-17T04:33:59.121866+0000","last_unstale":"2024-09-17T04:34:01.134091+0000","last_undegraded":"2024-09-17T04:34:01.134091+0000","last_fullsized":"2024-09-17T04:34:01.134091+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:33.592661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134630+0000","last_change":"2024-09-17T04:34:00.140363+0000","last_active":"2024-09-17T04:34:01.134630+0000","last_peered":"2024-09-17T04:34:01.134630+0000","last_clean":"2024-09-17T04:34:01.134630+0000","last_became_active":"2024-09-17T04:34:00.138809+0000","last_became_peered":"2024-09-17T04:34:00.138809+0000","last_unstale":"2024-09-17T04:34:01.134630+0000","last_undegraded":"2024-09-17T04:34:01.134630+0000","last_fullsized":"2024-09-17T04:34:01.134630+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:50.605107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542965+0000","last_change":"2024-09-17T04:33:59.118082+0000","last_active":"2024-09-17T04:34:01.542965+0000","last_peered":"2024-09-17T04:34:01.542965+0000","last_clean":"2024-09-17T04:34:01.542965+0000","last_became_active":"2024-09-17T04:33:59.117749+0000","last_became_peered":"2024-09-17T04:33:59.117749+0000","last_unstale":"2024-09-17T04:34:01.542965+0000","last_undegraded":"2024-09-17T04:34:01.542965+0000","last_fullsized":"2024-09-17T04:34:01.542965+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:22:09.267634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.536532+0000","last_change":"2024-09-17T04:34:00.143589+0000","last_active":"2024-09-17T04:34:01.536532+0000","last_peered":"2024-09-17T04:34:01.536532+0000","last_clean":"2024-09-17T04:34:01.536532+0000","last_became_active":"2024-09-17T04:34:00.143288+0000","last_became_peered":"2024-09-17T04:34:00.143288+0000","last_unstale":"2024-09-17T04:34:01.536532+0000","last_undegraded":"2024-09-17T04:34:01.536532+0000","last_fullsized":"2024-09-17T04:34:01.536532+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:43:32.412802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134818+0000","last_change":"2024-09-17T04:33:59.109283+0000","last_active":"2024-09-17T04:34:01.134818+0000","last_peered":"2024-09-17T04:34:01.134818+0000","last_clean":"2024-09-17T04:34:01.134818+0000","last_became_active":"2024-09-17T04:33:59.108953+0000","last_became_peered":"2024-09-17T04:33:59.108953+0000","last_unstale":"2024-09-17T04:34:01.134818+0000","last_undegraded":"2024-09-17T04:34:01.134818+0000","last_fullsized":"2024-09-17T04:34:01.134818+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:39.786291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542333+0000","last_change":"2024-09-17T04:34:00.138877+0000","last_active":"2024-09-17T04:34:01.542333+0000","last_peered":"2024-09-17T04:34:01.542333+0000","last_clean":"2024-09-17T04:34:01.542333+0000","last_became_active":"2024-09-17T04:34:00.138529+0000","last_became_peered":"2024-09-17T04:34:00.138529+0000","last_unstale":"2024-09-17T04:34:01.542333+0000","last_undegraded":"2024-09-17T04:34:01.542333+0000","last_fullsized":"2024-09-17T04:34:01.542333+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:05:34.986501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134665+0000","last_change":"2024-09-17T04:33:59.112268+0000","last_active":"2024-09-17T04:34:01.134665+0000","last_peered":"2024-09-17T04:34:01.134665+0000","last_clean":"2024-09-17T04:34:01.134665+0000","last_became_active":"2024-09-17T04:33:59.111822+0000","last_became_peered":"2024-09-17T04:33:59.111822+0000","last_unstale":"2024-09-17T04:34:01.134665+0000","last_undegraded":"2024-09-17T04:34:01.134665+0000","last_fullsized":"2024-09-17T04:34:01.134665+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:21:44.164301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134777+0000","last_change":"2024-09-17T04:34:00.131812+0000","last_active":"2024-09-17T04:34:01.134777+0000","last_peered":"2024-09-17T04:34:01.134777+0000","last_clean":"2024-09-17T04:34:01.134777+0000","last_became_active":"2024-09-17T04:34:00.131461+0000","last_became_peered":"2024-09-17T04:34:00.131461+0000","last_unstale":"2024-09-17T04:34:01.134777+0000","last_undegraded":"2024-09-17T04:34:01.134777+0000","last_fullsized":"2024-09-17T04:34:01.134777+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:08:13.529442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134780+0000","last_change":"2024-09-17T04:33:59.115490+0000","last_active":"2024-09-17T04:34:01.134780+0000","last_peered":"2024-09-17T04:34:01.134780+0000","last_clean":"2024-09-17T04:34:01.134780+0000","last_became_active":"2024-09-17T04:33:59.115105+0000","last_became_peered":"2024-09-17T04:33:59.115105+0000","last_unstale":"2024-09-17T04:34:01.134780+0000","last_undegraded":"2024-09-17T04:34:01.134780+0000","last_fullsized":"2024-09-17T04:34:01.134780+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:25.941127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.133761+0000","last_change":"2024-09-17T04:34:00.137771+0000","last_active":"2024-09-17T04:34:01.133761+0000","last_peered":"2024-09-17T04:34:01.133761+0000","last_clean":"2024-09-17T04:34:01.133761+0000","last_became_active":"2024-09-17T04:34:00.137424+0000","last_became_peered":"2024-09-17T04:34:00.137424+0000","last_unstale":"2024-09-17T04:34:01.133761+0000","last_undegraded":"2024-09-17T04:34:01.133761+0000","last_fullsized":"2024-09-17T04:34:01.133761+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:08:40.126582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542241+0000","last_change":"2024-09-17T04:33:59.116366+0000","last_active":"2024-09-17T04:34:01.542241+0000","last_peered":"2024-09-17T04:34:01.542241+0000","last_clean":"2024-09-17T04:34:01.542241+0000","last_became_active":"2024-09-17T04:33:59.116060+0000","last_became_peered":"2024-09-17T04:33:59.116060+0000","last_unstale":"2024-09-17T04:34:01.542241+0000","last_undegraded":"2024-09-17T04:34:01.542241+0000","last_fullsized":"2024-09-17T04:34:01.542241+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:23:51.532043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134654+0000","last_change":"2024-09-17T04:34:00.134391+0000","last_active":"2024-09-17T04:34:01.134654+0000","last_peered":"2024-09-17T04:34:01.134654+0000","last_clean":"2024-09-17T04:34:01.134654+0000","last_became_active":"2024-09-17T04:34:00.134057+0000","last_became_peered":"2024-09-17T04:34:00.134057+0000","last_unstale":"2024-09-17T04:34:01.134654+0000","last_undegraded":"2024-09-17T04:34:01.134654+0000","last_fullsized":"2024-09-17T04:34:01.134654+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:18.435735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.425093+0000","last_change":"2024-09-17T04:33:59.116483+0000","last_active":"2024-09-17T04:34:01.425093+0000","last_peered":"2024-09-17T04:34:01.425093+0000","last_clean":"2024-09-17T04:34:01.425093+0000","last_became_active":"2024-09-17T04:33:59.116144+0000","last_became_peered":"2024-09-17T04:33:59.116144+0000","last_unstale":"2024-09-17T04:34:01.425093+0000","last_undegraded":"2024-09-17T04:34:01.425093+0000","last_fullsized":"2024-09-17T04:34:01.425093+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:40.142964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.133680+0000","last_change":"2024-09-17T04:34:00.133387+0000","last_active":"2024-09-17T04:34:01.133680+0000","last_peered":"2024-09-17T04:34:01.133680+0000","last_clean":"2024-09-17T04:34:01.133680+0000","last_became_active":"2024-09-17T04:34:00.133075+0000","last_became_peered":"2024-09-17T04:34:00.133075+0000","last_unstale":"2024-09-17T04:34:01.133680+0000","last_undegraded":"2024-09-17T04:34:01.133680+0000","last_fullsized":"2024-09-17T04:34:01.133680+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:39.142294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134061+0000","last_change":"2024-09-17T04:33:59.116189+0000","last_active":"2024-09-17T04:34:01.134061+0000","last_peered":"2024-09-17T04:34:01.134061+0000","last_clean":"2024-09-17T04:34:01.134061+0000","last_became_active":"2024-09-17T04:33:59.115627+0000","last_became_peered":"2024-09-17T04:33:59.115627+0000","last_unstale":"2024-09-17T04:34:01.134061+0000","last_undegraded":"2024-09-17T04:34:01.134061+0000","last_fullsized":"2024-09-17T04:34:01.134061+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:31.619034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.541891+0000","last_change":"2024-09-17T04:34:00.134940+0000","last_active":"2024-09-17T04:34:01.541891+0000","last_peered":"2024-09-17T04:34:01.541891+0000","last_clean":"2024-09-17T04:34:01.541891+0000","last_became_active":"2024-09-17T04:34:00.134621+0000","last_became_peered":"2024-09-17T04:34:00.134621+0000","last_unstale":"2024-09-17T04:34:01.541891+0000","last_undegraded":"2024-09-17T04:34:01.541891+0000","last_fullsized":"2024-09-17T04:34:01.541891+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:02.203052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.535676+0000","last_change":"2024-09-17T04:33:59.113577+0000","last_active":"2024-09-17T04:34:01.535676+0000","last_peered":"2024-09-17T04:34:01.535676+0000","last_clean":"2024-09-17T04:34:01.535676+0000","last_became_active":"2024-09-17T04:33:59.113210+0000","last_became_peered":"2024-09-17T04:33:59.113210+0000","last_unstale":"2024-09-17T04:34:01.535676+0000","last_undegraded":"2024-09-17T04:34:01.535676+0000","last_fullsized":"2024-09-17T04:34:01.535676+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:06.764372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134718+0000","last_change":"2024-09-17T04:34:00.139613+0000","last_active":"2024-09-17T04:34:01.134718+0000","last_peered":"2024-09-17T04:34:01.134718+0000","last_clean":"2024-09-17T04:34:01.134718+0000","last_became_active":"2024-09-17T04:34:00.139292+0000","last_became_peered":"2024-09-17T04:34:00.139292+0000","last_unstale":"2024-09-17T04:34:01.134718+0000","last_undegraded":"2024-09-17T04:34:01.134718+0000","last_fullsized":"2024-09-17T04:34:01.134718+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:16:46.909065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.543031+0000","last_change":"2024-09-17T04:33:59.116203+0000","last_active":"2024-09-17T04:34:01.543031+0000","last_peered":"2024-09-17T04:34:01.543031+0000","last_clean":"2024-09-17T04:34:01.543031+0000","last_became_active":"2024-09-17T04:33:59.115805+0000","last_became_peered":"2024-09-17T04:33:59.115805+0000","last_unstale":"2024-09-17T04:34:01.543031+0000","last_undegraded":"2024-09-17T04:34:01.543031+0000","last_fullsized":"2024-09-17T04:34:01.543031+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:23:51.532043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.536523+0000","last_change":"2024-09-17T04:34:00.145064+0000","last_active":"2024-09-17T04:34:01.536523+0000","last_peered":"2024-09-17T04:34:01.536523+0000","last_clean":"2024-09-17T04:34:01.536523+0000","last_became_active":"2024-09-17T04:34:00.144786+0000","last_became_peered":"2024-09-17T04:34:00.144786+0000","last_unstale":"2024-09-17T04:34:01.536523+0000","last_undegraded":"2024-09-17T04:34:01.536523+0000","last_fullsized":"2024-09-17T04:34:01.536523+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:58.534910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:02.130019+0000","last_change":"2024-09-17T04:33:59.117234+0000","last_active":"2024-09-17T04:34:02.130019+0000","last_peered":"2024-09-17T04:34:02.130019+0000","last_clean":"2024-09-17T04:34:02.130019+0000","last_became_active":"2024-09-17T04:33:59.116890+0000","last_became_peered":"2024-09-17T04:33:59.116890+0000","last_unstale":"2024-09-17T04:34:02.130019+0000","last_undegraded":"2024-09-17T04:34:02.130019+0000","last_fullsized":"2024-09-17T04:34:02.130019+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:28.423942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542311+0000","last_change":"2024-09-17T04:34:00.144557+0000","last_active":"2024-09-17T04:34:01.542311+0000","last_peered":"2024-09-17T04:34:01.542311+0000","last_clean":"2024-09-17T04:34:01.542311+0000","last_became_active":"2024-09-17T04:34:00.144230+0000","last_became_peered":"2024-09-17T04:34:00.144230+0000","last_unstale":"2024-09-17T04:34:01.542311+0000","last_undegraded":"2024-09-17T04:34:01.542311+0000","last_fullsized":"2024-09-17T04:34:01.542311+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:46:25.341350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542538+0000","last_change":"2024-09-17T04:33:59.119426+0000","last_active":"2024-09-17T04:34:01.542538+0000","last_peered":"2024-09-17T04:34:01.542538+0000","last_clean":"2024-09-17T04:34:01.542538+0000","last_became_active":"2024-09-17T04:33:59.118894+0000","last_became_peered":"2024-09-17T04:33:59.118894+0000","last_unstale":"2024-09-17T04:34:01.542538+0000","last_undegraded":"2024-09-17T04:34:01.542538+0000","last_fullsized":"2024-09-17T04:34:01.542538+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:31.864330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.425276+0000","last_change":"2024-09-17T04:34:00.135067+0000","last_active":"2024-09-17T04:34:01.425276+0000","last_peered":"2024-09-17T04:34:01.425276+0000","last_clean":"2024-09-17T04:34:01.425276+0000","last_became_active":"2024-09-17T04:34:00.134742+0000","last_became_peered":"2024-09-17T04:34:00.134742+0000","last_unstale":"2024-09-17T04:34:01.425276+0000","last_undegraded":"2024-09-17T04:34:01.425276+0000","last_fullsized":"2024-09-17T04:34:01.425276+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:55.679577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542448+0000","last_change":"2024-09-17T04:33:59.114304+0000","last_active":"2024-09-17T04:34:01.542448+0000","last_peered":"2024-09-17T04:34:01.542448+0000","last_clean":"2024-09-17T04:34:01.542448+0000","last_became_active":"2024-09-17T04:33:59.113930+0000","last_became_peered":"2024-09-17T04:33:59.113930+0000","last_unstale":"2024-09-17T04:34:01.542448+0000","last_undegraded":"2024-09-17T04:34:01.542448+0000","last_fullsized":"2024-09-17T04:34:01.542448+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:11.433911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542543+0000","last_change":"2024-09-17T04:34:00.133293+0000","last_active":"2024-09-17T04:34:01.542543+0000","last_peered":"2024-09-17T04:34:01.542543+0000","last_clean":"2024-09-17T04:34:01.542543+0000","last_became_active":"2024-09-17T04:34:00.132910+0000","last_became_peered":"2024-09-17T04:34:00.132910+0000","last_unstale":"2024-09-17T04:34:01.542543+0000","last_undegraded":"2024-09-17T04:34:01.542543+0000","last_fullsized":"2024-09-17T04:34:01.542543+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:03.455957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542263+0000","last_change":"2024-09-17T04:33:59.119769+0000","last_active":"2024-09-17T04:34:01.542263+0000","last_peered":"2024-09-17T04:34:01.542263+0000","last_clean":"2024-09-17T04:34:01.542263+0000","last_became_active":"2024-09-17T04:33:59.119378+0000","last_became_peered":"2024-09-17T04:33:59.119378+0000","last_unstale":"2024-09-17T04:34:01.542263+0000","last_undegraded":"2024-09-17T04:34:01.542263+0000","last_fullsized":"2024-09-17T04:34:01.542263+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:02.777362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134147+0000","last_change":"2024-09-17T04:34:00.146323+0000","last_active":"2024-09-17T04:34:01.134147+0000","last_peered":"2024-09-17T04:34:01.134147+0000","last_clean":"2024-09-17T04:34:01.134147+0000","last_became_active":"2024-09-17T04:34:00.145997+0000","last_became_peered":"2024-09-17T04:34:00.145997+0000","last_unstale":"2024-09-17T04:34:01.134147+0000","last_undegraded":"2024-09-17T04:34:01.134147+0000","last_fullsized":"2024-09-17T04:34:01.134147+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:41.305573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134180+0000","last_change":"2024-09-17T04:33:59.109854+0000","last_active":"2024-09-17T04:34:01.134180+0000","last_peered":"2024-09-17T04:34:01.134180+0000","last_clean":"2024-09-17T04:34:01.134180+0000","last_became_active":"2024-09-17T04:33:59.109534+0000","last_became_peered":"2024-09-17T04:33:59.109534+0000","last_unstale":"2024-09-17T04:34:01.134180+0000","last_undegraded":"2024-09-17T04:34:01.134180+0000","last_fullsized":"2024-09-17T04:34:01.134180+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:24.369490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134207+0000","last_change":"2024-09-17T04:34:00.146380+0000","last_active":"2024-09-17T04:34:01.134207+0000","last_peered":"2024-09-17T04:34:01.134207+0000","last_clean":"2024-09-17T04:34:01.134207+0000","last_became_active":"2024-09-17T04:34:00.146024+0000","last_became_peered":"2024-09-17T04:34:00.146024+0000","last_unstale":"2024-09-17T04:34:01.134207+0000","last_undegraded":"2024-09-17T04:34:01.134207+0000","last_fullsized":"2024-09-17T04:34:01.134207+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:35:51.784535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.134211+0000","last_change":"2024-09-17T04:33:59.121607+0000","last_active":"2024-09-17T04:34:01.134211+0000","last_peered":"2024-09-17T04:34:01.134211+0000","last_clean":"2024-09-17T04:34:01.134211+0000","last_became_active":"2024-09-17T04:33:59.121262+0000","last_became_peered":"2024-09-17T04:33:59.121262+0000","last_unstale":"2024-09-17T04:34:01.134211+0000","last_undegraded":"2024-09-17T04:34:01.134211+0000","last_fullsized":"2024-09-17T04:34:01.134211+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:06.764372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.541922+0000","last_change":"2024-09-17T04:34:00.136520+0000","last_active":"2024-09-17T04:34:01.541922+0000","last_peered":"2024-09-17T04:34:01.541922+0000","last_clean":"2024-09-17T04:34:01.541922+0000","last_became_active":"2024-09-17T04:34:00.136200+0000","last_became_peered":"2024-09-17T04:34:00.136200+0000","last_unstale":"2024-09-17T04:34:01.541922+0000","last_undegraded":"2024-09-17T04:34:01.541922+0000","last_fullsized":"2024-09-17T04:34:01.541922+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:38:06.615802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.424981+0000","last_change":"2024-09-17T04:33:59.116928+0000","last_active":"2024-09-17T04:34:01.424981+0000","last_peered":"2024-09-17T04:34:01.424981+0000","last_clean":"2024-09-17T04:34:01.424981+0000","last_became_active":"2024-09-17T04:33:59.115764+0000","last_became_peered":"2024-09-17T04:33:59.115764+0000","last_unstale":"2024-09-17T04:34:01.424981+0000","last_undegraded":"2024-09-17T04:34:01.424981+0000","last_fullsized":"2024-09-17T04:34:01.424981+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:08.489333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":131,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T04:34:01.542243+0000","last_change":"2024-09-17T04:33:09.978742+0000","last_active":"2024-09-17T04:34:01.542243+0000","last_peered":"2024-09-17T04:34:01.542243+0000","last_clean":"2024-09-17T04:34:01.542243+0000","last_became_active":"2024-09-17T04:33:09.977496+0000","last_became_peered":"2024-09-17T04:33:09.977496+0000","last_unstale":"2024-09-17T04:34:01.542243+0000","last_undegraded":"2024-09-17T04:34:01.542243+0000","last_fullsized":"2024-09-17T04:34:01.542243+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:08.930340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:08.930340+0000","last_clean_scrub_stamp":"2024-09-17T04:33:08.930340+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:21:26.566102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542159,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":966656,"data_stored":732558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":376832,"data_stored":142716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542160,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":376832,"data_stored":142772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542159,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":142204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":385024,"data_stored":143734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542159,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":372736,"data_stored":142170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542158,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":385024,"data_stored":143734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542159,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":962560,"data_stored":732468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:34:04.569 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T04:34:04.570 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T04:34:04.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.729+0000 7f7a983eb640 1 -- 172.21.15.37:0/496251098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a900b34e0 msgr2=0x7f7a900d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:04.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.730+0000 7f7a983eb640 1 --2- 172.21.15.37:0/496251098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a900b34e0 0x7f7a900d0f20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7a900b40e0 tx=0x7f7a8002a4d0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.730+0000 7f7a983eb640 1 -- 172.21.15.37:0/496251098 shutdown_connections 2024-09-17T04:34:04.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.730+0000 7f7a983eb640 1 --2- 172.21.15.37:0/496251098 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f7a900d1460 0x7f7a900b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.730+0000 7f7a983eb640 1 --2- 172.21.15.37:0/496251098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a900b34e0 0x7f7a900d0f20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.730+0000 7f7a983eb640 1 --2- 172.21.15.37:0/496251098 >> v2:172.21.15.92:3300/0 conn(0x7f7a900b9ca0 0x7f7a900d2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.730+0000 7f7a983eb640 1 -- 172.21.15.37:0/496251098 >> 172.21.15.37:0/496251098 conn(0x7f7a9008bfb0 msgr2=0x7f7a9009d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:04.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.730+0000 7f7a983eb640 1 -- 172.21.15.37:0/496251098 shutdown_connections 2024-09-17T04:34:04.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.730+0000 7f7a983eb640 1 -- 172.21.15.37:0/496251098 wait complete. 2024-09-17T04:34:04.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.731+0000 7f7a983eb640 1 Processor -- start 2024-09-17T04:34:04.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.732+0000 7f7a983eb640 1 -- start start 2024-09-17T04:34:04.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.733+0000 7f7a983eb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a900b34e0 0x7f7a901f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:04.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.733+0000 7f7a983eb640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f7a900d1460 0x7f7a9015ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:04.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.733+0000 7f7a983eb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a901eb450 0x7f7a901eb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:04.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.733+0000 7f7a983eb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7a900b6230 con 0x7f7a900b34e0 2024-09-17T04:34:04.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.733+0000 7f7a983eb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7a900b5640 con 0x7f7a901eb450 2024-09-17T04:34:04.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.733+0000 7f7a983eb640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f7a900b63b0 con 0x7f7a900d1460 2024-09-17T04:34:04.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.733+0000 7f7a96187640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a900b34e0 0x7f7a901f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:04.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.733+0000 7f7a96187640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a900b34e0 0x7f7a901f5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35004/0 (socket says 172.21.15.37:35004) 2024-09-17T04:34:04.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.733+0000 7f7a96187640 1 -- 172.21.15.37:0/462685018 learned_addr learned my addr 172.21.15.37:0/462685018 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:04.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.733+0000 7f7a96988640 1 --2- 172.21.15.37:0/462685018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a901eb450 0x7f7a901eb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:04.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.733+0000 7f7a95986640 1 --2- 172.21.15.37:0/462685018 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f7a900d1460 0x7f7a9015ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:04.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.733+0000 7f7a96187640 1 -- 172.21.15.37:0/462685018 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f7a900d1460 msgr2=0x7f7a9015ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:04.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.733+0000 7f7a96187640 1 --2- 172.21.15.37:0/462685018 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f7a900d1460 0x7f7a9015ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.733+0000 7f7a96187640 1 -- 172.21.15.37:0/462685018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a901eb450 msgr2=0x7f7a901eb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:04.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.733+0000 7f7a96187640 1 --2- 172.21.15.37:0/462685018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a901eb450 0x7f7a901eb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:04.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.733+0000 7f7a96187640 1 -- 172.21.15.37:0/462685018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a900ccbc0 con 0x7f7a900b34e0 2024-09-17T04:34:04.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.733+0000 7f7a96187640 1 --2- 172.21.15.37:0/462685018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a900b34e0 0x7f7a901f5e70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00ea80 tx=0x7f7a8c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:04.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.734+0000 7f7a86ffd640 1 -- 172.21.15.37:0/462685018 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c00ca90 con 0x7f7a900b34e0 2024-09-17T04:34:04.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.734+0000 7f7a86ffd640 1 -- 172.21.15.37:0/462685018 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a8c00cd20 con 0x7f7a900b34e0 2024-09-17T04:34:04.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.734+0000 7f7a983eb640 1 -- 172.21.15.37:0/462685018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a901ebda0 con 0x7f7a900b34e0 2024-09-17T04:34:04.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.734+0000 7f7a86ffd640 1 -- 172.21.15.37:0/462685018 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c003bb0 con 0x7f7a900b34e0 2024-09-17T04:34:04.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.734+0000 7f7a983eb640 1 -- 172.21.15.37:0/462685018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a900c8160 con 0x7f7a900b34e0 2024-09-17T04:34:04.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.735+0000 7f7a983eb640 1 -- 172.21.15.37:0/462685018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a900b3060 con 0x7f7a900b34e0 2024-09-17T04:34:04.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.735+0000 7f7a86ffd640 1 -- 172.21.15.37:0/462685018 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7a8c004020 con 0x7f7a900b34e0 2024-09-17T04:34:04.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.736+0000 7f7a86ffd640 1 --2- 172.21.15.37:0/462685018 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f7a74078b90 0x7f7a7407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:04.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.736+0000 7f7a86ffd640 1 -- 172.21.15.37:0/462685018 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f7a8c09bb70 con 0x7f7a900b34e0 2024-09-17T04:34:04.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.736+0000 7f7a95986640 1 --2- 172.21.15.37:0/462685018 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f7a74078b90 0x7f7a7407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:04.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.737+0000 7f7a95986640 1 --2- 172.21.15.37:0/462685018 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f7a74078b90 0x7f7a7407b0a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7a80033000 tx=0x7f7a80004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:04.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.740+0000 7f7a86ffd640 1 -- 172.21.15.37:0/462685018 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f7a900b3060 con 0x7f7a900b34e0 2024-09-17T04:34:04.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:04.997+0000 7f7a983eb640 1 -- 172.21.15.37:0/462685018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f7a900b9ca0 con 0x7f7a900b34e0 2024-09-17T04:34:05.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.000+0000 7f7a86ffd640 1 -- 172.21.15.37:0/462685018 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7f7a901ebda0 con 0x7f7a900b34e0 2024-09-17T04:34:05.001 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-09-17T04:34:05.001 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBRBulm3vQYERAA/gAxWQECQvkbphM8KrWgUQ== 2024-09-17T04:34:05.001 INFO:teuthology.orchestra.run.smithi037.stdout: caps mds = "allow" 2024-09-17T04:34:05.001 INFO:teuthology.orchestra.run.smithi037.stdout: caps mon = "allow r" 2024-09-17T04:34:05.001 INFO:teuthology.orchestra.run.smithi037.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T04:34:05.001 INFO:teuthology.orchestra.run.smithi037.stderr:updated caps for client.0 2024-09-17T04:34:05.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.002+0000 7f7a983eb640 1 -- 172.21.15.37:0/462685018 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f7a74078b90 msgr2=0x7f7a7407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:05.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.002+0000 7f7a983eb640 1 --2- 172.21.15.37:0/462685018 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f7a74078b90 0x7f7a7407b0a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7a80033000 tx=0x7f7a80004730 comp rx=0 tx=0).stop 2024-09-17T04:34:05.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.002+0000 7f7a983eb640 1 -- 172.21.15.37:0/462685018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a900b34e0 msgr2=0x7f7a901f5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:05.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.002+0000 7f7a983eb640 1 --2- 172.21.15.37:0/462685018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a900b34e0 0x7f7a901f5e70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00ea80 tx=0x7f7a8c00ef40 comp rx=0 tx=0).stop 2024-09-17T04:34:05.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.002+0000 7f7a983eb640 1 -- 172.21.15.37:0/462685018 shutdown_connections 2024-09-17T04:34:05.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.002+0000 7f7a983eb640 1 --2- 172.21.15.37:0/462685018 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f7a74078b90 0x7f7a7407b0a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.002+0000 7f7a983eb640 1 --2- 172.21.15.37:0/462685018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a901eb450 0x7f7a901eb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.002+0000 7f7a983eb640 1 --2- 172.21.15.37:0/462685018 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f7a900d1460 0x7f7a9015ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.002+0000 7f7a983eb640 1 --2- 172.21.15.37:0/462685018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a900b34e0 0x7f7a901f5e70 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.002+0000 7f7a983eb640 1 -- 172.21.15.37:0/462685018 >> 172.21.15.37:0/462685018 conn(0x7f7a9008bfb0 msgr2=0x7f7a900b73e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:05.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.002+0000 7f7a983eb640 1 -- 172.21.15.37:0/462685018 shutdown_connections 2024-09-17T04:34:05.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.002+0000 7f7a983eb640 1 -- 172.21.15.37:0/462685018 wait complete. 2024-09-17T04:34:05.014 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:34:05.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.130+0000 7f563d615640 1 -- 172.21.15.37:0/79992482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56380d19c0 msgr2=0x7f56380d1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:05.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.130+0000 7f563d615640 1 --2- 172.21.15.37:0/79992482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56380d19c0 0x7f56380d1dd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5638074720 tx=0x7f562002a4a0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.130+0000 7f563d615640 1 -- 172.21.15.37:0/79992482 shutdown_connections 2024-09-17T04:34:05.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.130+0000 7f563d615640 1 --2- 172.21.15.37:0/79992482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56380d19c0 0x7f56380d1dd0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.130+0000 7f563d615640 1 --2- 172.21.15.37:0/79992482 >> v2:172.21.15.92:3300/0 conn(0x7f563815c630 0x7f56380cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.130+0000 7f563d615640 1 -- 172.21.15.37:0/79992482 >> 172.21.15.37:0/79992482 conn(0x7f56380cede0 msgr2=0x7f56380cf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:05.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.130+0000 7f563d615640 1 -- 172.21.15.37:0/79992482 shutdown_connections 2024-09-17T04:34:05.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.131+0000 7f563d615640 1 -- 172.21.15.37:0/79992482 wait complete. 2024-09-17T04:34:05.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.132+0000 7f563d615640 1 Processor -- start 2024-09-17T04:34:05.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.132+0000 7f563d615640 1 -- start start 2024-09-17T04:34:05.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.134+0000 7f563d615640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56380ccbc0 0x7f56380c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.134+0000 7f563d615640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f56380d19c0 0x7f5638151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.134+0000 7f563d615640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f563815c630 0x7f56380b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.134+0000 7f563d615640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f563815d280 con 0x7f56380ccbc0 2024-09-17T04:34:05.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.134+0000 7f563d615640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f563815d100 con 0x7f563815c630 2024-09-17T04:34:05.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.134+0000 7f563d615640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f563815d400 con 0x7f56380d19c0 2024-09-17T04:34:05.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.134+0000 7f5636ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56380ccbc0 0x7f56380c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.134+0000 7f5636ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56380ccbc0 0x7f56380c6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35030/0 (socket says 172.21.15.37:35030) 2024-09-17T04:34:05.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.134+0000 7f5636ffd640 1 -- 172.21.15.37:0/1194679268 learned_addr learned my addr 172.21.15.37:0/1194679268 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:05.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.134+0000 7f562e7fc640 1 --2- 172.21.15.37:0/1194679268 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f56380d19c0 0x7f5638151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.134+0000 7f56377fe640 1 --2- 172.21.15.37:0/1194679268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f563815c630 0x7f56380b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.134+0000 7f5636ffd640 1 -- 172.21.15.37:0/1194679268 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f56380d19c0 msgr2=0x7f5638151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:05.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.134+0000 7f5636ffd640 1 --2- 172.21.15.37:0/1194679268 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f56380d19c0 0x7f5638151860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.134+0000 7f5636ffd640 1 -- 172.21.15.37:0/1194679268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f563815c630 msgr2=0x7f56380b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:05.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.134+0000 7f5636ffd640 1 --2- 172.21.15.37:0/1194679268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f563815c630 0x7f56380b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.134+0000 7f5636ffd640 1 -- 172.21.15.37:0/1194679268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56380c69d0 con 0x7f56380ccbc0 2024-09-17T04:34:05.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.134+0000 7f5636ffd640 1 --2- 172.21.15.37:0/1194679268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56380ccbc0 0x7f56380c6320 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f562800d730 tx=0x7f562800dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:05.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.135+0000 7f562ffff640 1 -- 172.21.15.37:0/1194679268 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5628017070 con 0x7f56380ccbc0 2024-09-17T04:34:05.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.135+0000 7f562ffff640 1 -- 172.21.15.37:0/1194679268 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5628009a30 con 0x7f56380ccbc0 2024-09-17T04:34:05.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.135+0000 7f563d615640 1 -- 172.21.15.37:0/1194679268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56381f8700 con 0x7f56380ccbc0 2024-09-17T04:34:05.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.135+0000 7f562ffff640 1 -- 172.21.15.37:0/1194679268 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5628010460 con 0x7f56380ccbc0 2024-09-17T04:34:05.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.135+0000 7f563d615640 1 -- 172.21.15.37:0/1194679268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56381f89a0 con 0x7f56380ccbc0 2024-09-17T04:34:05.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.136+0000 7f563d615640 1 -- 172.21.15.37:0/1194679268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f563815d860 con 0x7f56380ccbc0 2024-09-17T04:34:05.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.137+0000 7f562ffff640 1 -- 172.21.15.37:0/1194679268 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f562801a070 con 0x7f56380ccbc0 2024-09-17T04:34:05.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.137+0000 7f562ffff640 1 --2- 172.21.15.37:0/1194679268 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f5618078b90 0x7f561807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.137+0000 7f562ffff640 1 -- 172.21.15.37:0/1194679268 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f562809adb0 con 0x7f56380ccbc0 2024-09-17T04:34:05.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.137+0000 7f562e7fc640 1 --2- 172.21.15.37:0/1194679268 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f5618078b90 0x7f561807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.138+0000 7f562e7fc640 1 --2- 172.21.15.37:0/1194679268 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f5618078b90 0x7f561807b0a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5620034000 tx=0x7f5620004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:05.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.142+0000 7f562ffff640 1 -- 172.21.15.37:0/1194679268 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f56381f8700 con 0x7f56380ccbc0 2024-09-17T04:34:05.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.367+0000 7f563d615640 1 -- 172.21.15.37:0/1194679268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56380d2af0 con 0x7f56380ccbc0 2024-09-17T04:34:05.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.368+0000 7f562ffff640 1 -- 172.21.15.37:0/1194679268 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5223 (secure 0 0 0) 0x7f5628062c30 con 0x7f56380ccbc0 2024-09-17T04:34:05.368 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:05.368 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"btime":"2024-09-17T04:34:03:138395+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4870,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/4152799414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4152799414},{"type":"v1","addr":"172.21.15.92:6836","nonce":4152799414}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4873,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/3730800721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":3730800721},{"type":"v1","addr":"172.21.15.92:6837","nonce":3730800721}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/1682438485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1682438485},{"type":"v1","addr":"172.21.15.37:6834","nonce":1682438485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:59.490265+0000","modified":"2024-09-17T04:34:02.518988+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1358600605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1358600605},{"type":"v1","addr":"172.21.15.37:6835","nonce":1358600605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4953,"qdb_cluster":[4953]},"id":2}]} 2024-09-17T04:34:05.368 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 17 2024-09-17T04:34:05.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.370+0000 7f563d615640 1 -- 172.21.15.37:0/1194679268 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f5618078b90 msgr2=0x7f561807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:05.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.370+0000 7f563d615640 1 --2- 172.21.15.37:0/1194679268 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f5618078b90 0x7f561807b0a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5620034000 tx=0x7f5620004730 comp rx=0 tx=0).stop 2024-09-17T04:34:05.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.370+0000 7f563d615640 1 -- 172.21.15.37:0/1194679268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56380ccbc0 msgr2=0x7f56380c6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:05.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.370+0000 7f563d615640 1 --2- 172.21.15.37:0/1194679268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56380ccbc0 0x7f56380c6320 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f562800d730 tx=0x7f562800dbf0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.371+0000 7f563d615640 1 -- 172.21.15.37:0/1194679268 shutdown_connections 2024-09-17T04:34:05.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.371+0000 7f563d615640 1 --2- 172.21.15.37:0/1194679268 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f5618078b90 0x7f561807b0a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.371+0000 7f563d615640 1 --2- 172.21.15.37:0/1194679268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f563815c630 0x7f56380b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.371+0000 7f563d615640 1 --2- 172.21.15.37:0/1194679268 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f56380d19c0 0x7f5638151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.371+0000 7f563d615640 1 --2- 172.21.15.37:0/1194679268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56380ccbc0 0x7f56380c6320 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.371+0000 7f563d615640 1 -- 172.21.15.37:0/1194679268 >> 172.21.15.37:0/1194679268 conn(0x7f56380cede0 msgr2=0x7f5638152820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:05.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.371+0000 7f563d615640 1 -- 172.21.15.37:0/1194679268 shutdown_connections 2024-09-17T04:34:05.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.371+0000 7f563d615640 1 -- 172.21.15.37:0/1194679268 wait complete. 2024-09-17T04:34:05.786 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T04:33:59.490265+0000', 'modified': '2024-09-17T04:34:02.518988+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4953}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4953': {'gid': 4953, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.37:6835/1358600605', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6833', 'nonce': 1358600605}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 1358600605}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4953, 'qdb_cluster': [4953]} 2024-09-17T04:34:05.786 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T04:34:05.786 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-17T04:34:05.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.908+0000 7fc9f5f28640 1 -- 172.21.15.37:0/2225446565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9f00d0ff0 msgr2=0x7fc9f00d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:05.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.908+0000 7fc9f5f28640 1 --2- 172.21.15.37:0/2225446565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9f00d0ff0 0x7fc9f00d1400 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc9f00d5690 tx=0x7fc9e802a820 comp rx=0 tx=0).stop 2024-09-17T04:34:05.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.908+0000 7fc9f5f28640 1 -- 172.21.15.37:0/2225446565 shutdown_connections 2024-09-17T04:34:05.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.908+0000 7fc9f5f28640 1 --2- 172.21.15.37:0/2225446565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9f00d0ff0 0x7fc9f00d1400 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.908+0000 7fc9f5f28640 1 --2- 172.21.15.37:0/2225446565 >> v2:172.21.15.92:3300/0 conn(0x7fc9f015fbe0 0x7fc9f0163040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.908+0000 7fc9f5f28640 1 -- 172.21.15.37:0/2225446565 >> 172.21.15.37:0/2225446565 conn(0x7fc9f006de00 msgr2=0x7fc9f006e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:05.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.908+0000 7fc9f5f28640 1 -- 172.21.15.37:0/2225446565 shutdown_connections 2024-09-17T04:34:05.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.908+0000 7fc9f5f28640 1 -- 172.21.15.37:0/2225446565 wait complete. 2024-09-17T04:34:05.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.910+0000 7fc9f5f28640 1 Processor -- start 2024-09-17T04:34:05.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.910+0000 7fc9f5f28640 1 -- start start 2024-09-17T04:34:05.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.911+0000 7fc9f5f28640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9f00cf7e0 0x7fc9f01f5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.911+0000 7fc9f5f28640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc9f00d0ff0 0x7fc9f0153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.911+0000 7fc9f5f28640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9f015fbe0 0x7fc9f00c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.911+0000 7fc9f5f28640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f00d32a0 con 0x7fc9f00cf7e0 2024-09-17T04:34:05.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.911+0000 7fc9f5f28640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f00d3120 con 0x7fc9f015fbe0 2024-09-17T04:34:05.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.911+0000 7fc9f5f28640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fc9f00d3420 con 0x7fc9f00d0ff0 2024-09-17T04:34:05.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.911+0000 7fc9ef7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9f00cf7e0 0x7fc9f01f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.912+0000 7fc9ef7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9f00cf7e0 0x7fc9f01f5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52630/0 (socket says 172.21.15.37:52630) 2024-09-17T04:34:05.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.912+0000 7fc9ef7fe640 1 -- 172.21.15.37:0/2568777959 learned_addr learned my addr 172.21.15.37:0/2568777959 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:05.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.912+0000 7fc9effff640 1 --2- 172.21.15.37:0/2568777959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9f015fbe0 0x7fc9f00c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.912+0000 7fc9e6ffd640 1 --2- 172.21.15.37:0/2568777959 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc9f00d0ff0 0x7fc9f0153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.912+0000 7fc9ef7fe640 1 -- 172.21.15.37:0/2568777959 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc9f00d0ff0 msgr2=0x7fc9f0153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:05.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.912+0000 7fc9ef7fe640 1 --2- 172.21.15.37:0/2568777959 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc9f00d0ff0 0x7fc9f0153c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.912+0000 7fc9ef7fe640 1 -- 172.21.15.37:0/2568777959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9f015fbe0 msgr2=0x7fc9f00c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:05.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.912+0000 7fc9ef7fe640 1 --2- 172.21.15.37:0/2568777959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9f015fbe0 0x7fc9f00c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.912+0000 7fc9ef7fe640 1 -- 172.21.15.37:0/2568777959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f0158ce0 con 0x7fc9f00cf7e0 2024-09-17T04:34:05.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.912+0000 7fc9ef7fe640 1 --2- 172.21.15.37:0/2568777959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9f00cf7e0 0x7fc9f01f5820 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000c970 tx=0x7fc9e000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:05.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.912+0000 7fc9ecff9640 1 -- 172.21.15.37:0/2568777959 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0007bb0 con 0x7fc9f00cf7e0 2024-09-17T04:34:05.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.912+0000 7fc9ecff9640 1 -- 172.21.15.37:0/2568777959 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9e0007e40 con 0x7fc9f00cf7e0 2024-09-17T04:34:05.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.912+0000 7fc9f5f28640 1 -- 172.21.15.37:0/2568777959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f0158fc0 con 0x7fc9f00cf7e0 2024-09-17T04:34:05.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.912+0000 7fc9ecff9640 1 -- 172.21.15.37:0/2568777959 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e000f690 con 0x7fc9f00cf7e0 2024-09-17T04:34:05.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.912+0000 7fc9f5f28640 1 -- 172.21.15.37:0/2568777959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f0159580 con 0x7fc9f00cf7e0 2024-09-17T04:34:05.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.914+0000 7fc9ecff9640 1 -- 172.21.15.37:0/2568777959 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc9e000f830 con 0x7fc9f00cf7e0 2024-09-17T04:34:05.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.915+0000 7fc9ecff9640 1 --2- 172.21.15.37:0/2568777959 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc9d0078bc0 0x7fc9d007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.915+0000 7fc9ecff9640 1 -- 172.21.15.37:0/2568777959 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc9e009b3e0 con 0x7fc9f00cf7e0 2024-09-17T04:34:05.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.915+0000 7fc9e6ffd640 1 --2- 172.21.15.37:0/2568777959 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc9d0078bc0 0x7fc9d007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.915+0000 7fc9e6ffd640 1 --2- 172.21.15.37:0/2568777959 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc9d0078bc0 0x7fc9d007b0d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0162d10 tx=0x7fc9d8007e90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:05.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.916+0000 7fc9fce94740 1 Processor -- start 2024-09-17T04:34:05.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.916+0000 7fc9fce94740 1 -- start start 2024-09-17T04:34:05.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.917+0000 7fc9fce94740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56016527a270 0x56016564bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.917+0000 7fc9fce94740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601652513b0 0x56016564c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.917+0000 7fc9fce94740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560165259590 0x560165669400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.917+0000 7fc9fce94740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560165594820 con 0x560165259590 2024-09-17T04:34:05.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.917+0000 7fc9fce94740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560165583c50 con 0x5601652513b0 2024-09-17T04:34:05.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.917+0000 7fc9fce94740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x560165587a60 con 0x56016527a270 2024-09-17T04:34:05.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.917+0000 7fc9effff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560165259590 0x560165669400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.917+0000 7fc9effff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560165259590 0x560165669400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52640/0 (socket says 172.21.15.37:52640) 2024-09-17T04:34:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.917+0000 7fc9effff640 1 -- 172.21.15.37:0/632220515 learned_addr learned my addr 172.21.15.37:0/632220515 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.918+0000 7fc9ef7fe640 1 --2- 172.21.15.37:0/632220515 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56016527a270 0x56016564bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.918+0000 7fc9e6ffd640 1 --2- 172.21.15.37:0/632220515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601652513b0 0x56016564c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.918+0000 7fc9effff640 1 -- 172.21.15.37:0/632220515 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56016527a270 msgr2=0x56016564bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.918+0000 7fc9effff640 1 --2- 172.21.15.37:0/632220515 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56016527a270 0x56016564bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.918+0000 7fc9effff640 1 -- 172.21.15.37:0/632220515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601652513b0 msgr2=0x56016564c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.918+0000 7fc9effff640 1 --2- 172.21.15.37:0/632220515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601652513b0 0x56016564c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.918+0000 7fc9effff640 1 -- 172.21.15.37:0/632220515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601652e5850 con 0x560165259590 2024-09-17T04:34:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.918+0000 7fc9effff640 1 --2- 172.21.15.37:0/632220515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560165259590 0x560165669400 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8035040 tx=0x7fc9e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39a530f03200877 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.918+0000 7fc9e57fa640 1 -- 172.21.15.37:0/632220515 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8037040 con 0x560165259590 2024-09-17T04:34:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.918+0000 7fc9e57fa640 1 -- 172.21.15.37:0/632220515 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9e8003dc0 con 0x560165259590 2024-09-17T04:34:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.918+0000 7fc9e57fa640 1 -- 172.21.15.37:0/632220515 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e80042a0 con 0x560165259590 2024-09-17T04:34:05.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.919+0000 7fc9fce94740 1 -- 172.21.15.37:0/632220515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560165259590 msgr2=0x560165669400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:05.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.919+0000 7fc9fce94740 1 --2- 172.21.15.37:0/632220515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560165259590 0x560165669400 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8035040 tx=0x7fc9e8004a40 comp rx=0 tx=0).stop 2024-09-17T04:34:05.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.919+0000 7fc9fce94740 1 -- 172.21.15.37:0/632220515 shutdown_connections 2024-09-17T04:34:05.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.919+0000 7fc9fce94740 1 --2- 172.21.15.37:0/632220515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560165259590 0x560165669400 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.919+0000 7fc9fce94740 1 --2- 172.21.15.37:0/632220515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601652513b0 0x56016564c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.919+0000 7fc9fce94740 1 --2- 172.21.15.37:0/632220515 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56016527a270 0x56016564bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.919+0000 7fc9fce94740 1 -- 172.21.15.37:0/632220515 >> 172.21.15.37:0/632220515 conn(0x56016526e3c0 msgr2=0x560165622280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:05.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.919+0000 7fc9fce94740 1 -- 172.21.15.37:0/632220515 shutdown_connections 2024-09-17T04:34:05.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.919+0000 7fc9fce94740 1 -- 172.21.15.37:0/632220515 wait complete. 2024-09-17T04:34:05.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.920+0000 7fc9fce94740 1 -- start start 2024-09-17T04:34:05.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.920+0000 7fc9fce94740 1 Processor -- start 2024-09-17T04:34:05.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.921+0000 7fc9fce94740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601652513b0 0x560165664600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.921+0000 7fc9fce94740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x560165259590 0x560165676260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.921+0000 7fc9fce94740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56016527a270 0x5601656794d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.921+0000 7fc9fce94740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5601652f4620 con 0x56016527a270 2024-09-17T04:34:05.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.921+0000 7fc9fce94740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5601655295f0 con 0x5601652513b0 2024-09-17T04:34:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.921+0000 7fc9fce94740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x56016564ca40 con 0x560165259590 2024-09-17T04:34:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.921+0000 7fc9fce94740 10 client.0 _ll_register_callbacks cb 0x56016552d0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:34:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.922+0000 7fc9effff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56016527a270 0x5601656794d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.922+0000 7fc9effff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56016527a270 0x5601656794d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52650/0 (socket says 172.21.15.37:52650) 2024-09-17T04:34:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.922+0000 7fc9effff640 1 -- 172.21.15.37:0/2912868998 learned_addr learned my addr 172.21.15.37:0/2912868998 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.922+0000 7fc9ef7fe640 1 --2- 172.21.15.37:0/2912868998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601652513b0 0x560165664600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.922+0000 7fc9e6ffd640 1 --2- 172.21.15.37:0/2912868998 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x560165259590 0x560165676260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.922+0000 7fc9effff640 1 -- 172.21.15.37:0/2912868998 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x560165259590 msgr2=0x560165676260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.922+0000 7fc9effff640 1 --2- 172.21.15.37:0/2912868998 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x560165259590 0x560165676260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.922+0000 7fc9effff640 1 -- 172.21.15.37:0/2912868998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601652513b0 msgr2=0x560165664600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.922+0000 7fc9effff640 1 --2- 172.21.15.37:0/2912868998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601652513b0 0x560165664600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.922+0000 7fc9effff640 1 -- 172.21.15.37:0/2912868998 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56016535c6b0 con 0x56016527a270 2024-09-17T04:34:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.922+0000 7fc9effff640 1 --2- 172.21.15.37:0/2912868998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56016527a270 0x5601656794d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc9e802ae10 tx=0x7fc9e8004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c2b5c675528b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.922+0000 7fc9ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-17T04:34:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.922+0000 7fc9ce7fc640 1 -- 172.21.15.37:0/2912868998 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8037030 con 0x56016527a270 2024-09-17T04:34:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.922+0000 7fc9ce7fc640 1 -- 172.21.15.37:0/2912868998 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc9e8032540 con 0x56016527a270 2024-09-17T04:34:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.922+0000 7fc9fce94740 1 -- 172.21.15.37:0/2912868998 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9e8003dc0 con 0x56016527a270 2024-09-17T04:34:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.922+0000 7fc9ce7fc640 1 -- 172.21.15.37:0/2912868998 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fc9e8055050 con 0x56016527a270 2024-09-17T04:34:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.922+0000 7fc9fce94740 10 client.5121 fetch_fsmap learned FSMap version 17 2024-09-17T04:34:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.922+0000 7fc9fce94740 1 -- 172.21.15.37:0/2912868998 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x560165664b40 con 0x56016527a270 2024-09-17T04:34:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.923+0000 7fc9ce7fc640 1 -- 172.21.15.37:0/2912868998 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9e8003dc0 con 0x56016527a270 2024-09-17T04:34:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.923+0000 7fc9ce7fc640 1 -- 172.21.15.37:0/2912868998 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7fc9e8058020 con 0x56016527a270 2024-09-17T04:34:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.923+0000 7fc9fce94740 10 client.5121 fetch_fsmap finished waiting for FSMap version 17 2024-09-17T04:34:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.923+0000 7fc9fce94740 10 client.5121 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T04:34:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.923+0000 7fc9fce94740 20 client.5121 populate_metadata read hostname 'smithi037' 2024-09-17T04:34:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.923+0000 7fc9fce94740 1 --2- 172.21.15.37:0/2912868998 >> [v2:172.21.15.37:6833/1358600605,v1:172.21.15.37:6835/1358600605] conn(0x560165238e40 0x56016562c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.923+0000 7fc9fce94740 4 client.5121 mds_command: new command op to 4953 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T04:34:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.923+0000 7fc9fce94740 1 -- 172.21.15.37:0/2912868998 --> [v2:172.21.15.37:6833/1358600605,v1:172.21.15.37:6835/1358600605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560165596770 con 0x560165238e40 2024-09-17T04:34:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.923+0000 7fc9ef7fe640 1 --2- 172.21.15.37:0/2912868998 >> [v2:172.21.15.37:6833/1358600605,v1:172.21.15.37:6835/1358600605] conn(0x560165238e40 0x56016562c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.923+0000 7fc9ce7fc640 10 client.5121 ms_handle_connect on v2:172.21.15.37:6833/1358600605 2024-09-17T04:34:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.923+0000 7fc9ef7fe640 1 --2- 172.21.15.37:0/2912868998 >> [v2:172.21.15.37:6833/1358600605,v1:172.21.15.37:6835/1358600605] conn(0x560165238e40 0x56016562c780 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7693d2cdebd85ed server_cookie=d6506f161c1bcaae in_seq=0 out_seq=0 2024-09-17T04:34:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.924+0000 7fc9ce7fc640 1 -- 172.21.15.37:0/2912868998 <== mds.0 v2:172.21.15.37:6833/1358600605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560165596770 con 0x560165238e40 2024-09-17T04:34:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.924+0000 7fc9ce7fc640 10 client.5121 handle_command_reply: tid=0 2024-09-17T04:34:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.924+0000 7fc9fce94740 1 client.5121 shutdown 2024-09-17T04:34:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.924+0000 7fc9fce94740 1 -- 172.21.15.37:0/2912868998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56016527a270 msgr2=0x5601656794d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.924+0000 7fc9fce94740 1 --2- 172.21.15.37:0/2912868998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56016527a270 0x5601656794d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc9e802ae10 tx=0x7fc9e8004440 comp rx=0 tx=0).stop 2024-09-17T04:34:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.924+0000 7fc9fce94740 1 -- 172.21.15.37:0/2912868998 shutdown_connections 2024-09-17T04:34:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.924+0000 7fc9fce94740 1 --2- 172.21.15.37:0/2912868998 >> [v2:172.21.15.37:6833/1358600605,v1:172.21.15.37:6835/1358600605] conn(0x560165238e40 0x56016562c780 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.924+0000 7fc9fce94740 1 --2- 172.21.15.37:0/2912868998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56016527a270 0x5601656794d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.924+0000 7fc9ce7fc640 0 client.5121 ms_handle_reset on v2:172.21.15.37:6833/1358600605 2024-09-17T04:34:05.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.924+0000 7fc9fce94740 1 --2- 172.21.15.37:0/2912868998 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x560165259590 0x560165676260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.924+0000 7fc9fce94740 1 --2- 172.21.15.37:0/2912868998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5601652513b0 0x560165664600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.924+0000 7fc9fce94740 1 -- 172.21.15.37:0/2912868998 >> 172.21.15.37:0/2912868998 conn(0x56016526e3c0 msgr2=0x56016561cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:05.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.924+0000 7fc9fce94740 1 -- 172.21.15.37:0/2912868998 shutdown_connections 2024-09-17T04:34:05.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.924+0000 7fc9fce94740 1 -- 172.21.15.37:0/2912868998 wait complete. 2024-09-17T04:34:05.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.925+0000 7fc9fce94740 20 client.5121 trim_cache size 0 max 16384 2024-09-17T04:34:05.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.943+0000 7fc9fce94740 1 Processor -- start 2024-09-17T04:34:05.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.943+0000 7fc9fce94740 1 -- start start 2024-09-17T04:34:05.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.944+0000 7fc9fce94740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601652513b0 0x560165621550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.945+0000 7fc9fce94740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x560165259590 0x560165664dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.945+0000 7fc9fce94740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56016527a270 0x560165665310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.945+0000 7fc9fce94740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f015c0f0 con 0x5601652513b0 2024-09-17T04:34:05.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.945+0000 7fc9fce94740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f00c86e0 con 0x56016527a270 2024-09-17T04:34:05.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.945+0000 7fc9fce94740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fc9f015baa0 con 0x560165259590 2024-09-17T04:34:05.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.945+0000 7fc9ef7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601652513b0 0x560165621550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.945+0000 7fc9ef7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601652513b0 0x560165621550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52662/0 (socket says 172.21.15.37:52662) 2024-09-17T04:34:05.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.945+0000 7fc9ef7fe640 1 -- 172.21.15.37:0/1028424843 learned_addr learned my addr 172.21.15.37:0/1028424843 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:05.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.945+0000 7fc9effff640 1 --2- 172.21.15.37:0/1028424843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56016527a270 0x560165665310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.945+0000 7fc9e6ffd640 1 --2- 172.21.15.37:0/1028424843 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x560165259590 0x560165664dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.945+0000 7fc9ef7fe640 1 -- 172.21.15.37:0/1028424843 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x560165259590 msgr2=0x560165664dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:05.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.945+0000 7fc9ef7fe640 1 --2- 172.21.15.37:0/1028424843 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x560165259590 0x560165664dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.945+0000 7fc9ef7fe640 1 -- 172.21.15.37:0/1028424843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56016527a270 msgr2=0x560165665310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:05.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.945+0000 7fc9ef7fe640 1 --2- 172.21.15.37:0/1028424843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56016527a270 0x560165665310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.945+0000 7fc9ef7fe640 1 -- 172.21.15.37:0/1028424843 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560165668f30 con 0x5601652513b0 2024-09-17T04:34:05.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.945+0000 7fc9e6ffd640 1 --2- 172.21.15.37:0/1028424843 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x560165259590 0x560165664dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:05.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.945+0000 7fc9ef7fe640 1 --2- 172.21.15.37:0/1028424843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601652513b0 0x560165621550 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00998b0 tx=0x7fc9e00997e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b015a74bfb4ec4 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:05.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.945+0000 7fc9e4ff9640 1 -- 172.21.15.37:0/1028424843 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e006cb40 con 0x5601652513b0 2024-09-17T04:34:05.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.945+0000 7fc9e4ff9640 1 -- 172.21.15.37:0/1028424843 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9e006a0c0 con 0x5601652513b0 2024-09-17T04:34:05.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.945+0000 7fc9e4ff9640 1 -- 172.21.15.37:0/1028424843 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e005f4d0 con 0x5601652513b0 2024-09-17T04:34:05.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.946+0000 7fc9fce94740 1 -- 172.21.15.37:0/1028424843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601652513b0 msgr2=0x560165621550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:05.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.946+0000 7fc9fce94740 1 --2- 172.21.15.37:0/1028424843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601652513b0 0x560165621550 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00998b0 tx=0x7fc9e00997e0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.946+0000 7fc9fce94740 1 -- 172.21.15.37:0/1028424843 shutdown_connections 2024-09-17T04:34:05.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.946+0000 7fc9fce94740 1 --2- 172.21.15.37:0/1028424843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56016527a270 0x560165665310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.946+0000 7fc9fce94740 1 --2- 172.21.15.37:0/1028424843 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x560165259590 0x560165664dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.946+0000 7fc9fce94740 1 --2- 172.21.15.37:0/1028424843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601652513b0 0x560165621550 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.946+0000 7fc9fce94740 1 -- 172.21.15.37:0/1028424843 >> 172.21.15.37:0/1028424843 conn(0x56016526e3c0 msgr2=0x560165617960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:05.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.946+0000 7fc9fce94740 1 -- 172.21.15.37:0/1028424843 shutdown_connections 2024-09-17T04:34:05.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.946+0000 7fc9fce94740 1 -- 172.21.15.37:0/1028424843 wait complete. 2024-09-17T04:34:05.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.947+0000 7fc9fce94740 1 -- start start 2024-09-17T04:34:05.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.947+0000 7fc9fce94740 1 Processor -- start 2024-09-17T04:34:05.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.948+0000 7fc9fce94740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601652513b0 0x56016563b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.948+0000 7fc9fce94740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x560165259590 0x5601656201a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.948+0000 7fc9fce94740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56016527a270 0x5601656e3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.948+0000 7fc9fce94740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5601652f4620 con 0x5601652513b0 2024-09-17T04:34:05.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.948+0000 7fc9fce94740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5601655295f0 con 0x56016527a270 2024-09-17T04:34:05.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.948+0000 7fc9fce94740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x560165679b30 con 0x560165259590 2024-09-17T04:34:05.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.949+0000 7fc9fce94740 10 client.0 _ll_register_callbacks cb 0x7fc9e8003b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:34:05.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.949+0000 7fc9ef7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601652513b0 0x56016563b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.949+0000 7fc9ef7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601652513b0 0x56016563b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52672/0 (socket says 172.21.15.37:52672) 2024-09-17T04:34:05.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.949+0000 7fc9ef7fe640 1 -- 172.21.15.37:0/2202292319 learned_addr learned my addr 172.21.15.37:0/2202292319 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:05.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.949+0000 7fc9effff640 1 --2- 172.21.15.37:0/2202292319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56016527a270 0x5601656e3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.949+0000 7fc9e6ffd640 1 --2- 172.21.15.37:0/2202292319 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x560165259590 0x5601656201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.949+0000 7fc9ef7fe640 1 -- 172.21.15.37:0/2202292319 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x560165259590 msgr2=0x5601656201a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:05.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.949+0000 7fc9ef7fe640 1 --2- 172.21.15.37:0/2202292319 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x560165259590 0x5601656201a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.949+0000 7fc9ef7fe640 1 -- 172.21.15.37:0/2202292319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56016527a270 msgr2=0x5601656e3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:05.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.949+0000 7fc9ef7fe640 1 --2- 172.21.15.37:0/2202292319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56016527a270 0x5601656e3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.949+0000 7fc9ef7fe640 1 -- 172.21.15.37:0/2202292319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601656e35d0 con 0x5601652513b0 2024-09-17T04:34:05.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.949+0000 7fc9e6ffd640 1 --2- 172.21.15.37:0/2202292319 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x560165259590 0x5601656201a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:05.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.949+0000 7fc9ef7fe640 1 --2- 172.21.15.37:0/2202292319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601652513b0 0x56016563b710 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0063420 tx=0x7fc9e005f8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa418ddce28eebb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:05.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.949+0000 7fc9ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-17T04:34:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.949+0000 7fc9ce7fc640 1 -- 172.21.15.37:0/2202292319 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0062d70 con 0x5601652513b0 2024-09-17T04:34:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.949+0000 7fc9ce7fc640 1 -- 172.21.15.37:0/2202292319 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc9e004aba0 con 0x5601652513b0 2024-09-17T04:34:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.949+0000 7fc9fce94740 1 -- 172.21.15.37:0/2202292319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9e006a0c0 con 0x5601652513b0 2024-09-17T04:34:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.949+0000 7fc9ce7fc640 1 -- 172.21.15.37:0/2202292319 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fc9e006cb40 con 0x5601652513b0 2024-09-17T04:34:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.949+0000 7fc9fce94740 10 client.5127 fetch_fsmap learned FSMap version 17 2024-09-17T04:34:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.949+0000 7fc9fce94740 1 -- 172.21.15.37:0/2202292319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5601656763b0 con 0x5601652513b0 2024-09-17T04:34:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.950+0000 7fc9ce7fc640 1 -- 172.21.15.37:0/2202292319 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9e006a0c0 con 0x5601652513b0 2024-09-17T04:34:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.950+0000 7fc9ce7fc640 1 -- 172.21.15.37:0/2202292319 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7fc9e0020030 con 0x5601652513b0 2024-09-17T04:34:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.950+0000 7fc9fce94740 10 client.5127 fetch_fsmap finished waiting for FSMap version 17 2024-09-17T04:34:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.950+0000 7fc9fce94740 10 client.5127 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T04:34:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.950+0000 7fc9fce94740 20 client.5127 populate_metadata read hostname 'smithi037' 2024-09-17T04:34:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.950+0000 7fc9fce94740 1 --2- 172.21.15.37:0/2202292319 >> [v2:172.21.15.37:6833/1358600605,v1:172.21.15.37:6835/1358600605] conn(0x56016564c3c0 0x5601656e9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.950+0000 7fc9fce94740 4 client.5127 mds_command: new command op to 4953 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T04:34:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.950+0000 7fc9fce94740 1 -- 172.21.15.37:0/2202292319 --> [v2:172.21.15.37:6833/1358600605,v1:172.21.15.37:6835/1358600605] -- command(tid 0: {"prefix": "status"}) -- 0x560165592220 con 0x56016564c3c0 2024-09-17T04:34:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.950+0000 7fc9e6ffd640 1 --2- 172.21.15.37:0/2202292319 >> [v2:172.21.15.37:6833/1358600605,v1:172.21.15.37:6835/1358600605] conn(0x56016564c3c0 0x5601656e9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.950+0000 7fc9ce7fc640 10 client.5127 ms_handle_connect on v2:172.21.15.37:6833/1358600605 2024-09-17T04:34:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.950+0000 7fc9e6ffd640 1 --2- 172.21.15.37:0/2202292319 >> [v2:172.21.15.37:6833/1358600605,v1:172.21.15.37:6835/1358600605] conn(0x56016564c3c0 0x5601656e9250 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f18e9c7acd4ff678 server_cookie=e3112d97ed4f6f57 in_seq=0 out_seq=0 2024-09-17T04:34:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.951+0000 7fc9ce7fc640 1 -- 172.21.15.37:0/2202292319 <== mds.0 v2:172.21.15.37:6833/1358600605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x560165592220 con 0x56016564c3c0 2024-09-17T04:34:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.951+0000 7fc9ce7fc640 10 client.5127 handle_command_reply: tid=0 2024-09-17T04:34:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.951+0000 7fc9fce94740 1 client.5127 shutdown 2024-09-17T04:34:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.951+0000 7fc9fce94740 1 -- 172.21.15.37:0/2202292319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601652513b0 msgr2=0x56016563b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.951+0000 7fc9fce94740 1 --2- 172.21.15.37:0/2202292319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601652513b0 0x56016563b710 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0063420 tx=0x7fc9e005f8b0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.951+0000 7fc9fce94740 1 -- 172.21.15.37:0/2202292319 shutdown_connections 2024-09-17T04:34:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.951+0000 7fc9fce94740 1 --2- 172.21.15.37:0/2202292319 >> [v2:172.21.15.37:6833/1358600605,v1:172.21.15.37:6835/1358600605] conn(0x56016564c3c0 0x5601656e9250 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.951+0000 7fc9fce94740 1 --2- 172.21.15.37:0/2202292319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56016527a270 0x5601656e3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.951+0000 7fc9fce94740 1 --2- 172.21.15.37:0/2202292319 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x560165259590 0x5601656201a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.951+0000 7fc9ce7fc640 0 client.5127 ms_handle_reset on v2:172.21.15.37:6833/1358600605 2024-09-17T04:34:05.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.951+0000 7fc9fce94740 1 --2- 172.21.15.37:0/2202292319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601652513b0 0x56016563b710 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.951+0000 7fc9fce94740 1 -- 172.21.15.37:0/2202292319 >> 172.21.15.37:0/2202292319 conn(0x56016526e3c0 msgr2=0x56016563aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:05.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.951+0000 7fc9fce94740 1 -- 172.21.15.37:0/2202292319 shutdown_connections 2024-09-17T04:34:05.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.951+0000 7fc9fce94740 1 -- 172.21.15.37:0/2202292319 wait complete. 2024-09-17T04:34:05.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.951+0000 7fc9fce94740 20 client.5127 trim_cache size 0 max 16384 2024-09-17T04:34:05.957 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T04:34:05.957 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "a908a603-430c-4209-8294-0c3460b35f42", 2024-09-17T04:34:05.957 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-09-17T04:34:05.957 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 4953, 2024-09-17T04:34:05.958 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-17T04:34:05.958 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-17T04:34:05.958 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-17T04:34:05.958 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 5.8345506729999999, 2024-09-17T04:34:05.958 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 17, 2024-09-17T04:34:05.958 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 33, 2024-09-17T04:34:05.958 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 33, 2024-09-17T04:34:05.958 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 12.761467185000001 2024-09-17T04:34:05.958 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T04:34:05.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.951+0000 7fc9f5f28640 1 -- 172.21.15.37:0/2568777959 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc9d0078bc0 msgr2=0x7fc9d007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:05.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.951+0000 7fc9f5f28640 1 --2- 172.21.15.37:0/2568777959 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc9d0078bc0 0x7fc9d007b0d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0162d10 tx=0x7fc9d8007e90 comp rx=0 tx=0).stop 2024-09-17T04:34:05.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.951+0000 7fc9f5f28640 1 -- 172.21.15.37:0/2568777959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9f00cf7e0 msgr2=0x7fc9f01f5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:05.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.951+0000 7fc9f5f28640 1 --2- 172.21.15.37:0/2568777959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9f00cf7e0 0x7fc9f01f5820 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000c970 tx=0x7fc9e000ce30 comp rx=0 tx=0).stop 2024-09-17T04:34:05.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.951+0000 7fc9f5f28640 1 -- 172.21.15.37:0/2568777959 shutdown_connections 2024-09-17T04:34:05.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.951+0000 7fc9f5f28640 1 --2- 172.21.15.37:0/2568777959 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc9d0078bc0 0x7fc9d007b0d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.951+0000 7fc9f5f28640 1 --2- 172.21.15.37:0/2568777959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9f015fbe0 0x7fc9f00c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.951+0000 7fc9f5f28640 1 --2- 172.21.15.37:0/2568777959 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc9f00d0ff0 0x7fc9f0153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.951+0000 7fc9f5f28640 1 --2- 172.21.15.37:0/2568777959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9f00cf7e0 0x7fc9f01f5820 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:05.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.951+0000 7fc9f5f28640 1 -- 172.21.15.37:0/2568777959 >> 172.21.15.37:0/2568777959 conn(0x7fc9f006de00 msgr2=0x7fc9f015dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:05.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.952+0000 7fc9f5f28640 1 -- 172.21.15.37:0/2568777959 shutdown_connections 2024-09-17T04:34:05.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:05.952+0000 7fc9f5f28640 1 -- 172.21.15.37:0/2568777959 wait complete. 2024-09-17T04:34:05.966 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:34:06.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.083+0000 7f43d541d640 1 -- 172.21.15.37:0/369740762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d00b6500 msgr2=0x7f43d01627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:06.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.083+0000 7f43d541d640 1 --2- 172.21.15.37:0/369740762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d00b6500 0x7f43d01627b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f43d00b6e10 tx=0x7f43c402a560 comp rx=0 tx=0).stop 2024-09-17T04:34:06.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.083+0000 7f43d541d640 1 -- 172.21.15.37:0/369740762 shutdown_connections 2024-09-17T04:34:06.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.083+0000 7f43d541d640 1 --2- 172.21.15.37:0/369740762 >> v2:172.21.15.92:3300/0 conn(0x7f43d0162cf0 0x7f43d006dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:06.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.083+0000 7f43d541d640 1 --2- 172.21.15.37:0/369740762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d00b6500 0x7f43d01627b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:06.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.083+0000 7f43d541d640 1 --2- 172.21.15.37:0/369740762 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f43d015e830 0x7f43d00b5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:06.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.083+0000 7f43d541d640 1 -- 172.21.15.37:0/369740762 >> 172.21.15.37:0/369740762 conn(0x7f43d00d4540 msgr2=0x7f43d00d4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:06.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.083+0000 7f43d541d640 1 -- 172.21.15.37:0/369740762 shutdown_connections 2024-09-17T04:34:06.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.083+0000 7f43d541d640 1 -- 172.21.15.37:0/369740762 wait complete. 2024-09-17T04:34:06.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.085+0000 7f43d541d640 1 Processor -- start 2024-09-17T04:34:06.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.085+0000 7f43d541d640 1 -- start start 2024-09-17T04:34:06.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.086+0000 7f43d541d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d00b6500 0x7f43d00b2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:06.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.086+0000 7f43d541d640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f43d015e830 0x7f43d014dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:06.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.086+0000 7f43d541d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43d0162cf0 0x7f43d014e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:06.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.086+0000 7f43d541d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43d006e630 con 0x7f43d00b6500 2024-09-17T04:34:06.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.086+0000 7f43d541d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f43d006e4b0 con 0x7f43d0162cf0 2024-09-17T04:34:06.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.086+0000 7f43d541d640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f43d006e7b0 con 0x7f43d015e830 2024-09-17T04:34:06.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.086+0000 7f43ceffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d00b6500 0x7f43d00b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:06.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.086+0000 7f43ceffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d00b6500 0x7f43d00b2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52700/0 (socket says 172.21.15.37:52700) 2024-09-17T04:34:06.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.086+0000 7f43ceffd640 1 -- 172.21.15.37:0/45467787 learned_addr learned my addr 172.21.15.37:0/45467787 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:06.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.086+0000 7f43ce7fc640 1 --2- 172.21.15.37:0/45467787 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f43d015e830 0x7f43d014dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:06.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.086+0000 7f43cf7fe640 1 --2- 172.21.15.37:0/45467787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43d0162cf0 0x7f43d014e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:06.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.087+0000 7f43ceffd640 1 -- 172.21.15.37:0/45467787 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f43d015e830 msgr2=0x7f43d014dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:06.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.087+0000 7f43ceffd640 1 --2- 172.21.15.37:0/45467787 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f43d015e830 0x7f43d014dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:06.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.087+0000 7f43ceffd640 1 -- 172.21.15.37:0/45467787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43d0162cf0 msgr2=0x7f43d014e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:06.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.087+0000 7f43ceffd640 1 --2- 172.21.15.37:0/45467787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43d0162cf0 0x7f43d014e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:06.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.087+0000 7f43ceffd640 1 -- 172.21.15.37:0/45467787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d00ca300 con 0x7f43d00b6500 2024-09-17T04:34:06.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.087+0000 7f43ce7fc640 1 --2- 172.21.15.37:0/45467787 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f43d015e830 0x7f43d014dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.087+0000 7f43cf7fe640 1 --2- 172.21.15.37:0/45467787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43d0162cf0 0x7f43d014e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.087+0000 7f43ceffd640 1 --2- 172.21.15.37:0/45467787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d00b6500 0x7f43d00b2710 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f43c000e7f0 tx=0x7f43c000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.087+0000 7f43b77fe640 1 -- 172.21.15.37:0/45467787 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c000eea0 con 0x7f43d00b6500 2024-09-17T04:34:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.087+0000 7f43b77fe640 1 -- 172.21.15.37:0/45467787 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43c000ccc0 con 0x7f43d00b6500 2024-09-17T04:34:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.087+0000 7f43d541d640 1 -- 172.21.15.37:0/45467787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d00ca5e0 con 0x7f43d00b6500 2024-09-17T04:34:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.087+0000 7f43b77fe640 1 -- 172.21.15.37:0/45467787 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0005280 con 0x7f43d00b6500 2024-09-17T04:34:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.087+0000 7f43d541d640 1 -- 172.21.15.37:0/45467787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d014d430 con 0x7f43d00b6500 2024-09-17T04:34:06.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.088+0000 7f43d541d640 1 -- 172.21.15.37:0/45467787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43d00b5fc0 con 0x7f43d00b6500 2024-09-17T04:34:06.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.089+0000 7f43b77fe640 1 -- 172.21.15.37:0/45467787 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f43c0005420 con 0x7f43d00b6500 2024-09-17T04:34:06.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.090+0000 7f43b77fe640 1 --2- 172.21.15.37:0/45467787 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f43bc078b90 0x7f43bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:06.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.090+0000 7f43b77fe640 1 -- 172.21.15.37:0/45467787 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f43c009b730 con 0x7f43d00b6500 2024-09-17T04:34:06.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.090+0000 7f43ce7fc640 1 --2- 172.21.15.37:0/45467787 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f43bc078b90 0x7f43bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:06.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.091+0000 7f43ce7fc640 1 --2- 172.21.15.37:0/45467787 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f43bc078b90 0x7f43bc07b0a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f43c4033000 tx=0x7f43c4004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:06.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.094+0000 7f43b77fe640 1 -- 172.21.15.37:0/45467787 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f43c0063b60 con 0x7f43d00b6500 2024-09-17T04:34:06.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.318+0000 7f43d541d640 1 -- 172.21.15.37:0/45467787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f43d00b9a90 con 0x7f43d00b6500 2024-09-17T04:34:06.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.319+0000 7f43b77fe640 1 -- 172.21.15.37:0/45467787 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f43c0063460 con 0x7f43d00b6500 2024-09-17T04:34:06.319 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:34:06.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.321+0000 7f43d541d640 1 -- 172.21.15.37:0/45467787 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f43bc078b90 msgr2=0x7f43bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:06.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.321+0000 7f43d541d640 1 --2- 172.21.15.37:0/45467787 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f43bc078b90 0x7f43bc07b0a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f43c4033000 tx=0x7f43c4004730 comp rx=0 tx=0).stop 2024-09-17T04:34:06.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.321+0000 7f43d541d640 1 -- 172.21.15.37:0/45467787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d00b6500 msgr2=0x7f43d00b2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:06.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.321+0000 7f43d541d640 1 --2- 172.21.15.37:0/45467787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d00b6500 0x7f43d00b2710 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f43c000e7f0 tx=0x7f43c000ecb0 comp rx=0 tx=0).stop 2024-09-17T04:34:06.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.321+0000 7f43d541d640 1 -- 172.21.15.37:0/45467787 shutdown_connections 2024-09-17T04:34:06.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.321+0000 7f43d541d640 1 --2- 172.21.15.37:0/45467787 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f43bc078b90 0x7f43bc07b0a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:06.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.321+0000 7f43d541d640 1 --2- 172.21.15.37:0/45467787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43d0162cf0 0x7f43d014e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:06.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.321+0000 7f43d541d640 1 --2- 172.21.15.37:0/45467787 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f43d015e830 0x7f43d014dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:06.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.321+0000 7f43d541d640 1 --2- 172.21.15.37:0/45467787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d00b6500 0x7f43d00b2710 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:06.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.321+0000 7f43d541d640 1 -- 172.21.15.37:0/45467787 >> 172.21.15.37:0/45467787 conn(0x7f43d00d4540 msgr2=0x7f43d00d4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:06.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.321+0000 7f43d541d640 1 -- 172.21.15.37:0/45467787 shutdown_connections 2024-09-17T04:34:06.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06.321+0000 7f43d541d640 1 -- 172.21.15.37:0/45467787 wait complete. 2024-09-17T04:34:06.332 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T04:34:06.332 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:34:06.332 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-09-17T04:34:06.333 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T04:34:06.333 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T04:34:06.333 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T04:34:06.333 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T04:34:06.333 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:34:06.333 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:34:06.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:06.333 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-09-17T04:34:06.350 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:34:06.350 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:34:06.350 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:34:06.350 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:06.350 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-09-17T04:34:06.350 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:06.350 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:34:06.350 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:34:06.350 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-09-17T04:34:06.351 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:34:06.351 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:34:06.351 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-09-17T04:34:06.351 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:34:06.351 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:34:06.351 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-09-17T04:34:06.351 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-17T04:34:06.351 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42705sec preferred_lft 42705sec 2024-09-17T04:34:06.351 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-09-17T04:34:06.351 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:06.351 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:34:06.351 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:34:06.351 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-09-17T04:34:06.351 INFO:teuthology.orchestra.run.smithi037.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:34:06.352 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 12:28:aa:73:a5:d1 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:34:06.352 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:34:06.352 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:06.352 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::8fb:ceff:fe9f:ed7d/64 scope link 2024-09-17T04:34:06.352 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:06.352 INFO:teuthology.orchestra.run.smithi037.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:34:06.352 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether d6:3e:49:21:b4:0b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:34:06.352 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::d43e:49ff:fe21:b40b/64 scope link 2024-09-17T04:34:06.352 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:06.352 INFO:teuthology.orchestra.run.smithi037.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:34:06.352 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 12:28:aa:73:a5:d1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:34:06.352 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::1028:aaff:fe73:a5d1/64 scope link 2024-09-17T04:34:06.353 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:06.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:06.353 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:34:06.414 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-09-17T04:34:06.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:06.416 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-09-17T04:34:06.472 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-09-17T04:34:06.473 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:34:06.473 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-17T04:34:06.473 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-17T04:34:06.473 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:34:06.474 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:34:06.474 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-17T04:34:06.474 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T04:34:06.474 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T04:34:06.475 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T04:34:06.475 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-17T04:34:06.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:34:06.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:34:06.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:06.602 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-09-17T04:34:06.618 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T04:34:06.618 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T04:34:06.618 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:34:06.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:06.618 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-09-17T04:34:06.684 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-17T04:34:06.684 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:34:06.684 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T04:34:06.740 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:34:06.740 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:34:06.740 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:34:06.795 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-09-17T04:34:06.862 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T04:34:06.918 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-09-17T04:34:06.918 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-09-17T04:34:06.918 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-09-17T04:34:06.918 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-09-17T04:34:06.919 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-09-17T04:34:06.919 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-09-17T04:34:06.919 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-09-17T04:34:06.919 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-09-17T04:34:06.919 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-09-17T04:34:06.919 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-09-17T04:34:06.919 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-09-17T04:34:06.919 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/selinux 2024-09-17T04:34:06.919 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T04:34:06.919 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-09-17T04:34:06.919 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-09-17T04:34:06.919 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-09-17T04:34:06.919 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-09-17T04:34:06.920 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-09-17T04:34:06.920 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-09-17T04:34:06.920 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T04:34:06.920 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T04:34:06.920 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T04:34:06.920 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T04:34:06.920 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-09-17T04:34:06.920 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-09-17T04:34:06.920 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T04:34:06.920 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T04:34:06.920 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T04:34:06.920 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T04:34:06.920 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-09-17T04:34:06.921 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:34:06.921 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:34:06.921 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:34:06.921 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:34:06.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:06.921 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-09-17T04:34:06.974 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T04:34:06.974 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-17T04:34:07.017 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-09-17T04:34:07.042 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T04:34:07.101 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-09-17T04:34:07.101 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-09-17T04:34:07.101 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-09-17T04:34:07.101 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-09-17T04:34:07.101 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-09-17T04:34:07.101 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-09-17T04:34:07.101 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-09-17T04:34:07.101 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-09-17T04:34:07.101 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-09-17T04:34:07.101 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-09-17T04:34:07.101 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-09-17T04:34:07.102 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/selinux 2024-09-17T04:34:07.102 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T04:34:07.102 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-09-17T04:34:07.102 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-09-17T04:34:07.102 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-09-17T04:34:07.102 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-09-17T04:34:07.102 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-09-17T04:34:07.102 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-09-17T04:34:07.102 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T04:34:07.102 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T04:34:07.102 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T04:34:07.102 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T04:34:07.102 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-09-17T04:34:07.102 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-09-17T04:34:07.102 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T04:34:07.103 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T04:34:07.103 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T04:34:07.103 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T04:34:07.103 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-09-17T04:34:07.103 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:34:07.103 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:34:07.103 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:34:07.103 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:34:07.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:07.103 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-09-17T04:34:07.112 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:2024-09-17T04:34:07.111+0000 7f7386a94480 -1 init, newargv = 0x7f7374001270 newargc=15 2024-09-17T04:34:07.112 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[48708]: starting ceph client 2024-09-17T04:34:07.117 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[48708]: starting fuse 2024-09-17T04:34:07.129 INFO:teuthology.orchestra.run.smithi037.stdout:42 2024-09-17T04:34:07.129 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-17T04:34:07.130 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-09-17T04:34:07.130 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-09-17T04:34:07.130 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-09-17T04:34:07.130 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-17T04:34:07.130 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-09-17T04:34:07.130 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-17T04:34:07.130 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-09-17T04:34:07.130 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-17T04:34:07.130 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-09-17T04:34:07.130 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-17T04:34:07.131 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-17T04:34:07.131 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-09-17T04:34:07.131 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-09-17T04:34:07.131 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-09-17T04:34:07.131 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-09-17T04:34:07.131 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-17T04:34:07.131 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-09-17T04:34:07.131 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T04:34:07.131 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T04:34:07.131 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T04:34:07.131 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-09-17T04:34:07.131 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-09-17T04:34:07.131 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-09-17T04:34:07.131 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T04:34:07.132 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-17T04:34:07.132 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.0")) 2024-09-17T04:34:07.132 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-17T04:34:07.251 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.0.48708.asok 2024-09-17T04:34:07.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:34:07.261 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48708.asok 2024-09-17T04:34:07.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:07.262 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48708.asok status 2024-09-17T04:34:07.407 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T04:34:07.407 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-09-17T04:34:07.407 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T04:34:07.407 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T04:34:07.407 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "0", 2024-09-17T04:34:07.408 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-09-17T04:34:07.408 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-17T04:34:07.408 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "48708", 2024-09-17T04:34:07.408 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-09-17T04:34:07.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:34:07.408 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-09-17T04:34:07.408 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-09-17T04:34:07.408 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 5142, 2024-09-17T04:34:07.408 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-09-17T04:34:07.408 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-09-17T04:34:07.408 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-09-17T04:34:07.409 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 5142 2024-09-17T04:34:07.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:34:07.409 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-09-17T04:34:07.409 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-09-17T04:34:07.409 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-09-17T04:34:07.409 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 281766257 2024-09-17T04:34:07.409 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:34:07.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:34:07.409 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-09-17T04:34:07.409 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-09-17T04:34:07.409 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-09-17T04:34:07.409 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 281766257 2024-09-17T04:34:07.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:34:07.409 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.5142 192.168.0.1:0/281766257", 2024-09-17T04:34:07.410 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.1:0/281766257", 2024-09-17T04:34:07.410 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-09-17T04:34:07.410 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 17, 2024-09-17T04:34:07.410 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 33, 2024-09-17T04:34:07.410 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-09-17T04:34:07.410 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-09-17T04:34:07.410 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-09-17T04:34:07.410 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T04:34:07.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:07.416 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-09-17T04:34:07.416 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-17T04:34:07.432 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-09-17T04:34:07.432 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-17T04:34:07.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:07.433 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:34:07.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:07.499 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2024-09-17T04:34:07.561 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:34:07.562 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:34:07.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:07.618 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=subdir/sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-09-17T04:34:07.722 INFO:teuthology.orchestra.run.smithi037.stderr:6+0 records in 2024-09-17T04:34:07.723 INFO:teuthology.orchestra.run.smithi037.stderr:6+0 records out 2024-09-17T04:34:07.723 INFO:teuthology.orchestra.run.smithi037.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0465774 s, 135 MB/s 2024-09-17T04:34:07.723 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-17T04:34:07.723 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-17T04:34:07.723 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-17T04:34:07.724 DEBUG:teuthology.orchestra.run.smithi037:> import json 2024-09-17T04:34:07.724 DEBUG:teuthology.orchestra.run.smithi037:> import sys 2024-09-17T04:34:07.724 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-17T04:34:07.724 DEBUG:teuthology.orchestra.run.smithi037:> try: 2024-09-17T04:34:07.724 DEBUG:teuthology.orchestra.run.smithi037:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2024-09-17T04:34:07.724 DEBUG:teuthology.orchestra.run.smithi037:> except OSError as e: 2024-09-17T04:34:07.724 DEBUG:teuthology.orchestra.run.smithi037:> sys.exit(e.errno) 2024-09-17T04:34:07.724 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-17T04:34:07.724 DEBUG:teuthology.orchestra.run.smithi037:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-09-17T04:34:07.724 DEBUG:teuthology.orchestra.run.smithi037:> print(json.dumps( 2024-09-17T04:34:07.724 DEBUG:teuthology.orchestra.run.smithi037:> dict([(a, getattr(s, a)) for a in attrs]), 2024-09-17T04:34:07.724 DEBUG:teuthology.orchestra.run.smithi037:> indent=2)) 2024-09-17T04:34:07.724 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-17T04:34:07.793 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T04:34:07.794 INFO:teuthology.orchestra.run.smithi037.stdout: "st_mode": 33188, 2024-09-17T04:34:07.794 INFO:teuthology.orchestra.run.smithi037.stdout: "st_ino": 1099511627777, 2024-09-17T04:34:07.794 INFO:teuthology.orchestra.run.smithi037.stdout: "st_dev": 42, 2024-09-17T04:34:07.794 INFO:teuthology.orchestra.run.smithi037.stdout: "st_nlink": 1, 2024-09-17T04:34:07.794 INFO:teuthology.orchestra.run.smithi037.stdout: "st_uid": 1000, 2024-09-17T04:34:07.794 INFO:teuthology.orchestra.run.smithi037.stdout: "st_gid": 1333, 2024-09-17T04:34:07.794 INFO:teuthology.orchestra.run.smithi037.stdout: "st_size": 6291456, 2024-09-17T04:34:07.794 INFO:teuthology.orchestra.run.smithi037.stdout: "st_atime": 1726547647.675734, 2024-09-17T04:34:07.794 INFO:teuthology.orchestra.run.smithi037.stdout: "st_mtime": 1726547647.7041082, 2024-09-17T04:34:07.795 INFO:teuthology.orchestra.run.smithi037.stdout: "st_ctime": 1726547647.7041082 2024-09-17T04:34:07.795 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T04:34:07.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:34:07.803 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:34:07.803 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:34:07.860 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:34:08.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.016+0000 7f987e762640 1 -- 172.21.15.37:0/2108238565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9860003b10 msgr2=0x7f9860003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:08.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.016+0000 7f987e762640 1 --2- 172.21.15.37:0/2108238565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9860003b10 0x7f9860003f80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f986800c310 tx=0x7f9860005830 comp rx=0 tx=0).stop 2024-09-17T04:34:08.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.016+0000 7f987ccff640 1 -- 172.21.15.37:0/2108238565 reap_dead start 2024-09-17T04:34:08.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.017+0000 7f987e762640 1 -- 172.21.15.37:0/2108238565 shutdown_connections 2024-09-17T04:34:08.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.017+0000 7f987e762640 1 -- 172.21.15.37:0/2108238565 >> 172.21.15.37:0/2108238565 conn(0x7f98780d4540 msgr2=0x7f98780d4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:08.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.017+0000 7f987e762640 1 -- 172.21.15.37:0/2108238565 shutdown_connections 2024-09-17T04:34:08.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.017+0000 7f987e762640 1 -- 172.21.15.37:0/2108238565 wait complete. 2024-09-17T04:34:08.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.018+0000 7f987e762640 1 Processor -- start 2024-09-17T04:34:08.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.019+0000 7f987e762640 1 -- start start 2024-09-17T04:34:08.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.020+0000 7f987e762640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9860003b10 0x7f98781570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:08.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.020+0000 7f987e762640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9878161350 0x7f987815ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:08.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.020+0000 7f987e762640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98781f88f0 0x7f98781f8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:08.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.020+0000 7f987e762640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9878162930 con 0x7f98781f88f0 2024-09-17T04:34:08.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.020+0000 7f987e762640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f98781627b0 con 0x7f9878161350 2024-09-17T04:34:08.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.020+0000 7f987e762640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f9878162ab0 con 0x7f9860003b10 2024-09-17T04:34:08.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.020+0000 7f987ccff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98781f88f0 0x7f98781f8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:08.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.020+0000 7f987ccff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98781f88f0 0x7f98781f8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52722/0 (socket says 172.21.15.37:52722) 2024-09-17T04:34:08.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.020+0000 7f987ccff640 1 -- 172.21.15.37:0/3669099950 learned_addr learned my addr 172.21.15.37:0/3669099950 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:08.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.020+0000 7f98777fe640 1 --2- 172.21.15.37:0/3669099950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9878161350 0x7f987815ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:08.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.020+0000 7f9877fff640 1 --2- 172.21.15.37:0/3669099950 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9860003b10 0x7f98781570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:08.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.020+0000 7f987ccff640 1 -- 172.21.15.37:0/3669099950 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9860003b10 msgr2=0x7f98781570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:08.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.020+0000 7f987ccff640 1 --2- 172.21.15.37:0/3669099950 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9860003b10 0x7f98781570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.020+0000 7f987ccff640 1 -- 172.21.15.37:0/3669099950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9878161350 msgr2=0x7f987815ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:08.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.020+0000 7f987ccff640 1 --2- 172.21.15.37:0/3669099950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9878161350 0x7f987815ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.020+0000 7f987ccff640 1 -- 172.21.15.37:0/3669099950 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9878157610 con 0x7f98781f88f0 2024-09-17T04:34:08.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.021+0000 7f987ccff640 1 --2- 172.21.15.37:0/3669099950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98781f88f0 0x7f98781f8d00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9860004780 tx=0x7f9870006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:08.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.021+0000 7f9874ff9640 1 -- 172.21.15.37:0/3669099950 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9870011900 con 0x7f98781f88f0 2024-09-17T04:34:08.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.021+0000 7f9874ff9640 1 -- 172.21.15.37:0/3669099950 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9870018070 con 0x7f98781f88f0 2024-09-17T04:34:08.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.021+0000 7f987e762640 1 -- 172.21.15.37:0/3669099950 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f987814a200 con 0x7f98781f88f0 2024-09-17T04:34:08.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.021+0000 7f9874ff9640 1 -- 172.21.15.37:0/3669099950 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9870011d40 con 0x7f98781f88f0 2024-09-17T04:34:08.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.021+0000 7f987e762640 1 -- 172.21.15.37:0/3669099950 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9878154760 con 0x7f98781f88f0 2024-09-17T04:34:08.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.022+0000 7f987e762640 1 -- 172.21.15.37:0/3669099950 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98780b5fc0 con 0x7f98781f88f0 2024-09-17T04:34:08.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.022+0000 7f9874ff9640 1 -- 172.21.15.37:0/3669099950 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f987001d460 con 0x7f98781f88f0 2024-09-17T04:34:08.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.023+0000 7f9874ff9640 1 --2- 172.21.15.37:0/3669099950 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9864078b90 0x7f986407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:08.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.023+0000 7f9874ff9640 1 -- 172.21.15.37:0/3669099950 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f987009b540 con 0x7f98781f88f0 2024-09-17T04:34:08.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.023+0000 7f9877fff640 1 --2- 172.21.15.37:0/3669099950 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9864078b90 0x7f986407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:08.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.024+0000 7f9877fff640 1 --2- 172.21.15.37:0/3669099950 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9864078b90 0x7f986407b0a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9868033000 tx=0x7f9868038040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:08.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.028+0000 7f9874ff9640 1 -- 172.21.15.37:0/3669099950 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f98700633c0 con 0x7f98781f88f0 2024-09-17T04:34:08.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.228+0000 7f987e762640 1 -- 172.21.15.37:0/3669099950 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f987815dc30 con 0x7f98781f88f0 2024-09-17T04:34:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.229+0000 7f9874ff9640 1 -- 172.21.15.37:0/3669099950 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) ==== 80+0+0 (secure 0 0 0) 0x7f9870067df0 con 0x7f98781f88f0 2024-09-17T04:34:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-09-17T04:34:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.231+0000 7f987e762640 1 -- 172.21.15.37:0/3669099950 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9864078b90 msgr2=0x7f986407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.231+0000 7f987e762640 1 --2- 172.21.15.37:0/3669099950 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9864078b90 0x7f986407b0a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9868033000 tx=0x7f9868038040 comp rx=0 tx=0).stop 2024-09-17T04:34:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.231+0000 7f987e762640 1 -- 172.21.15.37:0/3669099950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98781f88f0 msgr2=0x7f98781f8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.231+0000 7f987e762640 1 --2- 172.21.15.37:0/3669099950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98781f88f0 0x7f98781f8d00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9860004780 tx=0x7f9870006430 comp rx=0 tx=0).stop 2024-09-17T04:34:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.231+0000 7f987e762640 1 -- 172.21.15.37:0/3669099950 shutdown_connections 2024-09-17T04:34:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.231+0000 7f987e762640 1 --2- 172.21.15.37:0/3669099950 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9864078b90 0x7f986407b0a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.231+0000 7f987e762640 1 --2- 172.21.15.37:0/3669099950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98781f88f0 0x7f98781f8d00 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.231+0000 7f987e762640 1 --2- 172.21.15.37:0/3669099950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9878161350 0x7f987815ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.231+0000 7f987e762640 1 --2- 172.21.15.37:0/3669099950 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9860003b10 0x7f98781570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.231+0000 7f987e762640 1 -- 172.21.15.37:0/3669099950 >> 172.21.15.37:0/3669099950 conn(0x7f98780d4540 msgr2=0x7f98780b7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.231+0000 7f987e762640 1 -- 172.21.15.37:0/3669099950 shutdown_connections 2024-09-17T04:34:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.231+0000 7f987e762640 1 -- 172.21.15.37:0/3669099950 wait complete. 2024-09-17T04:34:08.244 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:34:08.244 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:34:08.260 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:34:08.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.416+0000 7f8bde4c5640 1 -- 172.21.15.37:0/2851350045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bd80d0ff0 msgr2=0x7f8bd80d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:08.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.416+0000 7f8bde4c5640 1 --2- 172.21.15.37:0/2851350045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bd80d0ff0 0x7f8bd80d1400 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0005fd0 tx=0x7f8bd002a8b0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.416+0000 7f8bde4c5640 1 -- 172.21.15.37:0/2851350045 shutdown_connections 2024-09-17T04:34:08.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.416+0000 7f8bde4c5640 1 --2- 172.21.15.37:0/2851350045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bd80d0ff0 0x7f8bd80d1400 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.416+0000 7f8bde4c5640 1 --2- 172.21.15.37:0/2851350045 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8bd8163040 0x7f8bd80cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.416+0000 7f8bde4c5640 1 -- 172.21.15.37:0/2851350045 >> 172.21.15.37:0/2851350045 conn(0x7f8bd806de00 msgr2=0x7f8bd806e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:08.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.416+0000 7f8bde4c5640 1 -- 172.21.15.37:0/2851350045 shutdown_connections 2024-09-17T04:34:08.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.416+0000 7f8bde4c5640 1 -- 172.21.15.37:0/2851350045 wait complete. 2024-09-17T04:34:08.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.418+0000 7f8bde4c5640 1 Processor -- start 2024-09-17T04:34:08.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.418+0000 7f8bde4c5640 1 -- start start 2024-09-17T04:34:08.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.419+0000 7f8bde4c5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bd80d0ff0 0x7f8bd80c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:08.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.419+0000 7f8bde4c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bd815fbe0 0x7f8bd814b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:08.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.419+0000 7f8bde4c5640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8bd8163040 0x7f8bd814bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:08.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.419+0000 7f8bde4c5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd80d32e0 con 0x7f8bd815fbe0 2024-09-17T04:34:08.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.419+0000 7f8bde4c5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd80d3160 con 0x7f8bd80d0ff0 2024-09-17T04:34:08.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.419+0000 7f8bde4c5640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f8bd80d3460 con 0x7f8bd8163040 2024-09-17T04:34:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.419+0000 7f8bd77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bd815fbe0 0x7f8bd814b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.419+0000 7f8bd77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bd815fbe0 0x7f8bd814b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52738/0 (socket says 172.21.15.37:52738) 2024-09-17T04:34:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.419+0000 7f8bd77fe640 1 -- 172.21.15.37:0/378454330 learned_addr learned my addr 172.21.15.37:0/378454330 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.420+0000 7f8bd7fff640 1 --2- 172.21.15.37:0/378454330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bd80d0ff0 0x7f8bd80c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.420+0000 7f8bd77fe640 1 -- 172.21.15.37:0/378454330 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8bd8163040 msgr2=0x7f8bd814bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.420+0000 7f8bdca62640 1 --2- 172.21.15.37:0/378454330 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8bd8163040 0x7f8bd814bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.420+0000 7f8bd77fe640 1 --2- 172.21.15.37:0/378454330 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8bd8163040 0x7f8bd814bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.420+0000 7f8bd77fe640 1 -- 172.21.15.37:0/378454330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bd80d0ff0 msgr2=0x7f8bd80c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.420+0000 7f8bd77fe640 1 --2- 172.21.15.37:0/378454330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bd80d0ff0 0x7f8bd80c9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.420+0000 7f8bd77fe640 1 -- 172.21.15.37:0/378454330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd80c89b0 con 0x7f8bd815fbe0 2024-09-17T04:34:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.420+0000 7f8bd7fff640 1 --2- 172.21.15.37:0/378454330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bd80d0ff0 0x7f8bd80c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.420+0000 7f8bdca62640 1 --2- 172.21.15.37:0/378454330 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8bd8163040 0x7f8bd814bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:08.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.420+0000 7f8bd77fe640 1 --2- 172.21.15.37:0/378454330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bd815fbe0 0x7f8bd814b2e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00d740 tx=0x7f8bcc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:08.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.420+0000 7f8bd4ff9640 1 -- 172.21.15.37:0/378454330 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc017070 con 0x7f8bd815fbe0 2024-09-17T04:34:08.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.421+0000 7f8bd4ff9640 1 -- 172.21.15.37:0/378454330 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8bcc009d20 con 0x7f8bd815fbe0 2024-09-17T04:34:08.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.421+0000 7f8bd4ff9640 1 -- 172.21.15.37:0/378454330 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc0106c0 con 0x7f8bd815fbe0 2024-09-17T04:34:08.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.421+0000 7f8bde4c5640 1 -- 172.21.15.37:0/378454330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd80c8c90 con 0x7f8bd815fbe0 2024-09-17T04:34:08.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.421+0000 7f8bde4c5640 1 -- 172.21.15.37:0/378454330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd80c9250 con 0x7f8bd815fbe0 2024-09-17T04:34:08.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.422+0000 7f8bde4c5640 1 -- 172.21.15.37:0/378454330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd80c8720 con 0x7f8bd815fbe0 2024-09-17T04:34:08.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.422+0000 7f8bd4ff9640 1 -- 172.21.15.37:0/378454330 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8bcc01a070 con 0x7f8bd815fbe0 2024-09-17T04:34:08.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.423+0000 7f8bd4ff9640 1 --2- 172.21.15.37:0/378454330 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8bc4078bc0 0x7f8bc407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:08.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.423+0000 7f8bd4ff9640 1 -- 172.21.15.37:0/378454330 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8bcc09af50 con 0x7f8bd815fbe0 2024-09-17T04:34:08.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.423+0000 7f8bd7fff640 1 --2- 172.21.15.37:0/378454330 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8bc4078bc0 0x7f8bc407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:08.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.424+0000 7f8bd7fff640 1 --2- 172.21.15.37:0/378454330 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8bc4078bc0 0x7f8bc407b0d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8000f60 tx=0x7f8bc8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:08.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.427+0000 7f8bd4ff9640 1 -- 172.21.15.37:0/378454330 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8bcc063380 con 0x7f8bd815fbe0 2024-09-17T04:34:08.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.625+0000 7f8bde4c5640 1 -- 172.21.15.37:0/378454330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8bd80d3980 con 0x7f8bd815fbe0 2024-09-17T04:34:08.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.626+0000 7f8bd4ff9640 1 -- 172.21.15.37:0/378454330 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) ==== 80+0+0 (secure 0 0 0) 0x7f8bcc062c80 con 0x7f8bd815fbe0 2024-09-17T04:34:08.627 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-09-17T04:34:08.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.628+0000 7f8bde4c5640 1 -- 172.21.15.37:0/378454330 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8bc4078bc0 msgr2=0x7f8bc407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:08.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.628+0000 7f8bde4c5640 1 --2- 172.21.15.37:0/378454330 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8bc4078bc0 0x7f8bc407b0d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8000f60 tx=0x7f8bc8009040 comp rx=0 tx=0).stop 2024-09-17T04:34:08.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.628+0000 7f8bde4c5640 1 -- 172.21.15.37:0/378454330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bd815fbe0 msgr2=0x7f8bd814b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:08.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.628+0000 7f8bde4c5640 1 --2- 172.21.15.37:0/378454330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bd815fbe0 0x7f8bd814b2e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00d740 tx=0x7f8bcc00dc00 comp rx=0 tx=0).stop 2024-09-17T04:34:08.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.628+0000 7f8bde4c5640 1 -- 172.21.15.37:0/378454330 shutdown_connections 2024-09-17T04:34:08.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.628+0000 7f8bde4c5640 1 --2- 172.21.15.37:0/378454330 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8bc4078bc0 0x7f8bc407b0d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.628+0000 7f8bde4c5640 1 --2- 172.21.15.37:0/378454330 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8bd8163040 0x7f8bd814bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.628+0000 7f8bde4c5640 1 --2- 172.21.15.37:0/378454330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bd815fbe0 0x7f8bd814b2e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.628+0000 7f8bde4c5640 1 --2- 172.21.15.37:0/378454330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bd80d0ff0 0x7f8bd80c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:08.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.628+0000 7f8bde4c5640 1 -- 172.21.15.37:0/378454330 >> 172.21.15.37:0/378454330 conn(0x7f8bd806de00 msgr2=0x7f8bd815e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:08.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.628+0000 7f8bde4c5640 1 -- 172.21.15.37:0/378454330 shutdown_connections 2024-09-17T04:34:08.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:08.628+0000 7f8bde4c5640 1 -- 172.21.15.37:0/378454330 wait complete. 2024-09-17T04:34:08.640 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-09-17T04:34:08.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:08.640 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-17T04:34:08.665 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[48708]: fuse finished with error 0 and tester_r 0 2024-09-17T04:34:08.667 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:34:14.389 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:34:14.392 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:34:14.768 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-09-17T04:34:14.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:14.769 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T04:34:14.789 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:34:14.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.954+0000 7fca7fd52640 1 -- 172.21.15.37:0/1286591628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca78162cf0 msgr2=0x7fca7806dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:14.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.954+0000 7fca7fd52640 1 --2- 172.21.15.37:0/1286591628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca78162cf0 0x7fca7806dd60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fca7806ef30 tx=0x7fca7002aa30 comp rx=0 tx=0).stop 2024-09-17T04:34:14.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.955+0000 7fca7fd52640 1 -- 172.21.15.37:0/1286591628 shutdown_connections 2024-09-17T04:34:14.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.955+0000 7fca7fd52640 1 --2- 172.21.15.37:0/1286591628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca78162cf0 0x7fca7806dd60 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:14.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.955+0000 7fca7fd52640 1 --2- 172.21.15.37:0/1286591628 >> v2:172.21.15.92:3300/0 conn(0x7fca780b6500 0x7fca781627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:14.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.955+0000 7fca7fd52640 1 --2- 172.21.15.37:0/1286591628 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fca7815e830 0x7fca780b5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:14.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.955+0000 7fca7fd52640 1 -- 172.21.15.37:0/1286591628 >> 172.21.15.37:0/1286591628 conn(0x7fca780d4540 msgr2=0x7fca780d4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:14.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.955+0000 7fca7fd52640 1 -- 172.21.15.37:0/1286591628 shutdown_connections 2024-09-17T04:34:14.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.955+0000 7fca7fd52640 1 -- 172.21.15.37:0/1286591628 wait complete. 2024-09-17T04:34:14.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.956+0000 7fca7fd52640 1 Processor -- start 2024-09-17T04:34:14.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.957+0000 7fca7fd52640 1 -- start start 2024-09-17T04:34:14.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.958+0000 7fca7fd52640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca780b6500 0x7fca780ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:14.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.958+0000 7fca7fd52640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fca7815e830 0x7fca781503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:14.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.958+0000 7fca7fd52640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca78162cf0 0x7fca7814d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:14.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.958+0000 7fca7fd52640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fca7806e630 con 0x7fca78162cf0 2024-09-17T04:34:14.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.958+0000 7fca7fd52640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fca7806e4b0 con 0x7fca780b6500 2024-09-17T04:34:14.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.958+0000 7fca7fd52640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fca7806e7b0 con 0x7fca7815e830 2024-09-17T04:34:14.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.958+0000 7fca7e2ef640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca78162cf0 0x7fca7814d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:14.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.958+0000 7fca7e2ef640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca78162cf0 0x7fca7814d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52764/0 (socket says 172.21.15.37:52764) 2024-09-17T04:34:14.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.958+0000 7fca7e2ef640 1 -- 172.21.15.37:0/64205086 learned_addr learned my addr 172.21.15.37:0/64205086 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:14.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.958+0000 7fca7daee640 1 --2- 172.21.15.37:0/64205086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca780b6500 0x7fca780ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:14.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.958+0000 7fca7d2ed640 1 --2- 172.21.15.37:0/64205086 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fca7815e830 0x7fca781503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:14.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.958+0000 7fca7e2ef640 1 -- 172.21.15.37:0/64205086 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fca7815e830 msgr2=0x7fca781503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:14.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.958+0000 7fca7e2ef640 1 --2- 172.21.15.37:0/64205086 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fca7815e830 0x7fca781503c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:14.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.958+0000 7fca7e2ef640 1 -- 172.21.15.37:0/64205086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca780b6500 msgr2=0x7fca780ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:14.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.958+0000 7fca7e2ef640 1 --2- 172.21.15.37:0/64205086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca780b6500 0x7fca780ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:14.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.958+0000 7fca7e2ef640 1 -- 172.21.15.37:0/64205086 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca780c9660 con 0x7fca78162cf0 2024-09-17T04:34:14.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.958+0000 7fca7daee640 1 --2- 172.21.15.37:0/64205086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca780b6500 0x7fca780ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:14.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.959+0000 7fca7e2ef640 1 --2- 172.21.15.37:0/64205086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca78162cf0 0x7fca7814d080 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fca7002b020 tx=0x7fca70003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:14.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.959+0000 7fca6e7fc640 1 -- 172.21.15.37:0/64205086 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca70033050 con 0x7fca78162cf0 2024-09-17T04:34:14.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.959+0000 7fca6e7fc640 1 -- 172.21.15.37:0/64205086 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca70004450 con 0x7fca78162cf0 2024-09-17T04:34:14.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.959+0000 7fca7fd52640 1 -- 172.21.15.37:0/64205086 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca780c9940 con 0x7fca78162cf0 2024-09-17T04:34:14.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.959+0000 7fca6e7fc640 1 -- 172.21.15.37:0/64205086 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca70004840 con 0x7fca78162cf0 2024-09-17T04:34:14.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.959+0000 7fca7fd52640 1 -- 172.21.15.37:0/64205086 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca780c9f00 con 0x7fca78162cf0 2024-09-17T04:34:14.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.960+0000 7fca7fd52640 1 -- 172.21.15.37:0/64205086 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca780c93d0 con 0x7fca78162cf0 2024-09-17T04:34:14.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.961+0000 7fca6e7fc640 1 -- 172.21.15.37:0/64205086 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fca70036070 con 0x7fca78162cf0 2024-09-17T04:34:14.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.961+0000 7fca6e7fc640 1 --2- 172.21.15.37:0/64205086 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fca5c078b90 0x7fca5c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:14.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.962+0000 7fca6e7fc640 1 -- 172.21.15.37:0/64205086 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fca700bc1f0 con 0x7fca78162cf0 2024-09-17T04:34:14.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.962+0000 7fca7daee640 1 --2- 172.21.15.37:0/64205086 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fca5c078b90 0x7fca5c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:14.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.962+0000 7fca7daee640 1 --2- 172.21.15.37:0/64205086 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fca5c078b90 0x7fca5c07b0a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fca7400ba60 tx=0x7fca74005cf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:14.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:14.966+0000 7fca6e7fc640 1 -- 172.21.15.37:0/64205086 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fca700846c0 con 0x7fca78162cf0 2024-09-17T04:34:15.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.191+0000 7fca7fd52640 1 -- 172.21.15.37:0/64205086 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca780b5fc0 con 0x7fca78162cf0 2024-09-17T04:34:15.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.192+0000 7fca6e7fc640 1 -- 172.21.15.37:0/64205086 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5223 (secure 0 0 0) 0x7fca70083fc0 con 0x7fca78162cf0 2024-09-17T04:34:15.193 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.193 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"btime":"2024-09-17T04:34:03:138395+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4870,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/4152799414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4152799414},{"type":"v1","addr":"172.21.15.92:6836","nonce":4152799414}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4873,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/3730800721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":3730800721},{"type":"v1","addr":"172.21.15.92:6837","nonce":3730800721}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/1682438485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1682438485},{"type":"v1","addr":"172.21.15.37:6834","nonce":1682438485}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:59.490265+0000","modified":"2024-09-17T04:34:02.518988+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1358600605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1358600605},{"type":"v1","addr":"172.21.15.37:6835","nonce":1358600605}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4953,"qdb_cluster":[4953]},"id":2}]} 2024-09-17T04:34:15.194 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 17 2024-09-17T04:34:15.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.194+0000 7fca7fd52640 1 -- 172.21.15.37:0/64205086 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fca5c078b90 msgr2=0x7fca5c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:15.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.194+0000 7fca7fd52640 1 --2- 172.21.15.37:0/64205086 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fca5c078b90 0x7fca5c07b0a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fca7400ba60 tx=0x7fca74005cf0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.194+0000 7fca7fd52640 1 -- 172.21.15.37:0/64205086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca78162cf0 msgr2=0x7fca7814d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:15.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.194+0000 7fca7fd52640 1 --2- 172.21.15.37:0/64205086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca78162cf0 0x7fca7814d080 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fca7002b020 tx=0x7fca70003f10 comp rx=0 tx=0).stop 2024-09-17T04:34:15.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.194+0000 7fca7fd52640 1 -- 172.21.15.37:0/64205086 shutdown_connections 2024-09-17T04:34:15.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.194+0000 7fca7fd52640 1 --2- 172.21.15.37:0/64205086 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fca5c078b90 0x7fca5c07b0a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.194+0000 7fca7fd52640 1 --2- 172.21.15.37:0/64205086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca78162cf0 0x7fca7814d080 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.194+0000 7fca7fd52640 1 --2- 172.21.15.37:0/64205086 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fca7815e830 0x7fca781503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.194+0000 7fca7fd52640 1 --2- 172.21.15.37:0/64205086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca780b6500 0x7fca780ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.194+0000 7fca7fd52640 1 -- 172.21.15.37:0/64205086 >> 172.21.15.37:0/64205086 conn(0x7fca780d4540 msgr2=0x7fca7815ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:15.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.195+0000 7fca7fd52640 1 -- 172.21.15.37:0/64205086 shutdown_connections 2024-09-17T04:34:15.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.195+0000 7fca7fd52640 1 -- 172.21.15.37:0/64205086 wait complete. 2024-09-17T04:34:15.207 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-09-17T04:34:15.341 INFO:teuthology.orchestra.run.smithi037.stdout:{"message":"","return_code":0} 2024-09-17T04:34:15.342 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-17T04:34:15.342 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T04:34:15.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.459+0000 7f39e3638640 1 -- 172.21.15.37:0/3845435729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39dc0cd6e0 msgr2=0x7f39dc0d0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:15.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.459+0000 7f39e3638640 1 --2- 172.21.15.37:0/3845435729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39dc0cd6e0 0x7f39dc0d0250 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009830 tx=0x7f39d402a480 comp rx=0 tx=0).stop 2024-09-17T04:34:15.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.459+0000 7f39e3638640 1 -- 172.21.15.37:0/3845435729 shutdown_connections 2024-09-17T04:34:15.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.459+0000 7f39e3638640 1 --2- 172.21.15.37:0/3845435729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39dc0cd6e0 0x7f39dc0d0250 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.459+0000 7f39e3638640 1 --2- 172.21.15.37:0/3845435729 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f39dc0b6910 0x7f39dc0cd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.459+0000 7f39e3638640 1 -- 172.21.15.37:0/3845435729 >> 172.21.15.37:0/3845435729 conn(0x7f39dc0cdbb0 msgr2=0x7f39dc0cdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:15.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.459+0000 7f39e3638640 1 -- 172.21.15.37:0/3845435729 shutdown_connections 2024-09-17T04:34:15.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.459+0000 7f39e3638640 1 -- 172.21.15.37:0/3845435729 wait complete. 2024-09-17T04:34:15.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.461+0000 7f39e3638640 1 Processor -- start 2024-09-17T04:34:15.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.461+0000 7f39e3638640 1 -- start start 2024-09-17T04:34:15.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.463+0000 7f39e3638640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f39dc074c40 0x7f39dc14c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:15.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.463+0000 7f39e3638640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39dc0b6910 0x7f39dc1f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:15.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.463+0000 7f39e3638640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39dc0cd6e0 0x7f39dc0c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:15.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.463+0000 7f39e3638640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc071210 con 0x7f39dc0b6910 2024-09-17T04:34:15.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.463+0000 7f39e3638640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc0d0b60 con 0x7f39dc0cd6e0 2024-09-17T04:34:15.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.463+0000 7f39e3638640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f39dc071390 con 0x7f39dc074c40 2024-09-17T04:34:15.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.463+0000 7f39e0bd3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39dc0b6910 0x7f39dc1f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:15.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.463+0000 7f39e0bd3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39dc0b6910 0x7f39dc1f75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38296/0 (socket says 172.21.15.37:38296) 2024-09-17T04:34:15.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.463+0000 7f39e0bd3640 1 -- 172.21.15.37:0/1459599407 learned_addr learned my addr 172.21.15.37:0/1459599407 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:15.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.463+0000 7f39e13d4640 1 --2- 172.21.15.37:0/1459599407 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f39dc074c40 0x7f39dc14c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:15.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.463+0000 7f39e1bd5640 1 --2- 172.21.15.37:0/1459599407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39dc0cd6e0 0x7f39dc0c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:15.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.463+0000 7f39e0bd3640 1 -- 172.21.15.37:0/1459599407 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f39dc074c40 msgr2=0x7f39dc14c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:15.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.463+0000 7f39e0bd3640 1 --2- 172.21.15.37:0/1459599407 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f39dc074c40 0x7f39dc14c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.463+0000 7f39e0bd3640 1 -- 172.21.15.37:0/1459599407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39dc0cd6e0 msgr2=0x7f39dc0c6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:15.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.463+0000 7f39e0bd3640 1 --2- 172.21.15.37:0/1459599407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39dc0cd6e0 0x7f39dc0c6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.463+0000 7f39e0bd3640 1 -- 172.21.15.37:0/1459599407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39dc1f8140 con 0x7f39dc0b6910 2024-09-17T04:34:15.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.463+0000 7f39e1bd5640 1 --2- 172.21.15.37:0/1459599407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39dc0cd6e0 0x7f39dc0c6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:15.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.463+0000 7f39e0bd3640 1 --2- 172.21.15.37:0/1459599407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39dc0b6910 0x7f39dc1f75d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f39d000d750 tx=0x7f39d000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:15.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.463+0000 7f39d9ffb640 1 -- 172.21.15.37:0/1459599407 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0017070 con 0x7f39dc0b6910 2024-09-17T04:34:15.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.463+0000 7f39d9ffb640 1 -- 172.21.15.37:0/1459599407 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39d0004020 con 0x7f39dc0b6910 2024-09-17T04:34:15.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.463+0000 7f39e3638640 1 -- 172.21.15.37:0/1459599407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39dc1f8420 con 0x7f39dc0b6910 2024-09-17T04:34:15.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.463+0000 7f39d9ffb640 1 -- 172.21.15.37:0/1459599407 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0004450 con 0x7f39dc0b6910 2024-09-17T04:34:15.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.464+0000 7f39e3638640 1 -- 172.21.15.37:0/1459599407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39dc0c8250 con 0x7f39dc0b6910 2024-09-17T04:34:15.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.465+0000 7f39e3638640 1 -- 172.21.15.37:0/1459599407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39dc1f7eb0 con 0x7f39dc0b6910 2024-09-17T04:34:15.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.466+0000 7f39d9ffb640 1 -- 172.21.15.37:0/1459599407 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f39d00099d0 con 0x7f39dc0b6910 2024-09-17T04:34:15.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.466+0000 7f39d9ffb640 1 --2- 172.21.15.37:0/1459599407 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f39c8078b90 0x7f39c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:15.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.467+0000 7f39d9ffb640 1 -- 172.21.15.37:0/1459599407 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f39d009b090 con 0x7f39dc0b6910 2024-09-17T04:34:15.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.467+0000 7f39e13d4640 1 --2- 172.21.15.37:0/1459599407 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f39c8078b90 0x7f39c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:15.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.467+0000 7f39e13d4640 1 --2- 172.21.15.37:0/1459599407 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f39c8078b90 0x7f39c807b0a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0012e0 tx=0x7f39cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:15.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.471+0000 7f39d9ffb640 1 -- 172.21.15.37:0/1459599407 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f39d0020020 con 0x7f39dc0b6910 2024-09-17T04:34:15.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.697+0000 7f39e3638640 1 -- 172.21.15.37:0/1459599407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f39dc0718b0 con 0x7f39dc0b6910 2024-09-17T04:34:15.698 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-09-17T04:34:15.698+0000 7f4732e82640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T04:34:15.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.699+0000 7f39d9ffb640 1 -- 172.21.15.37:0/1459599407 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7f39d00634e0 con 0x7f39dc0b6910 2024-09-17T04:34:15.700 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T04:34:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.703+0000 7f39e3638640 1 -- 172.21.15.37:0/1459599407 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f39c8078b90 msgr2=0x7f39c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.703+0000 7f39e3638640 1 --2- 172.21.15.37:0/1459599407 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f39c8078b90 0x7f39c807b0a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0012e0 tx=0x7f39cc009040 comp rx=0 tx=0).stop 2024-09-17T04:34:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.703+0000 7f39e3638640 1 -- 172.21.15.37:0/1459599407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39dc0b6910 msgr2=0x7f39dc1f75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.703+0000 7f39e3638640 1 --2- 172.21.15.37:0/1459599407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39dc0b6910 0x7f39dc1f75d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f39d000d750 tx=0x7f39d000dc10 comp rx=0 tx=0).stop 2024-09-17T04:34:15.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.703+0000 7f39e3638640 1 -- 172.21.15.37:0/1459599407 shutdown_connections 2024-09-17T04:34:15.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.703+0000 7f39e3638640 1 --2- 172.21.15.37:0/1459599407 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f39c8078b90 0x7f39c807b0a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.703+0000 7f39e3638640 1 --2- 172.21.15.37:0/1459599407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39dc0cd6e0 0x7f39dc0c6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.703+0000 7f39e3638640 1 --2- 172.21.15.37:0/1459599407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39dc0b6910 0x7f39dc1f75d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.703+0000 7f39e3638640 1 --2- 172.21.15.37:0/1459599407 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f39dc074c40 0x7f39dc14c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:15.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.703+0000 7f39e3638640 1 -- 172.21.15.37:0/1459599407 >> 172.21.15.37:0/1459599407 conn(0x7f39dc0cdbb0 msgr2=0x7f39dc1e73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:15.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.703+0000 7f39e3638640 1 -- 172.21.15.37:0/1459599407 shutdown_connections 2024-09-17T04:34:15.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:15.704+0000 7f39e3638640 1 -- 172.21.15.37:0/1459599407 wait complete. 2024-09-17T04:34:15.706 INFO:tasks.ceph.mds.a.smithi037.stderr: -3838> 2024-09-17T04:34:04.029+0000 7f8bf8a92640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T04:34:15.717 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T04:34:15.719 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T04:34:15.765 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-09-17T04:34:15.766 INFO:teuthology.orchestra.run.smithi037.stdout:[global] 2024-09-17T04:34:15.766 INFO:teuthology.orchestra.run.smithi037.stdout: chdir = "" 2024-09-17T04:34:15.766 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T04:34:15.766 INFO:teuthology.orchestra.run.smithi037.stdout: auth supported = cephx 2024-09-17T04:34:15.766 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.766 INFO:teuthology.orchestra.run.smithi037.stdout: filestore xattr use omap = true 2024-09-17T04:34:15.766 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.766 INFO:teuthology.orchestra.run.smithi037.stdout: mon clock drift allowed = 1.000 2024-09-17T04:34:15.766 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.766 INFO:teuthology.orchestra.run.smithi037.stdout: osd crush chooseleaf type = 0 2024-09-17T04:34:15.766 INFO:teuthology.orchestra.run.smithi037.stdout: auth debug = true 2024-09-17T04:34:15.766 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.766 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on old message = true 2024-09-17T04:34:15.767 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on bug = true 2024-09-17T04:34:15.767 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.767 INFO:teuthology.orchestra.run.smithi037.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T04:34:15.767 INFO:teuthology.orchestra.run.smithi037.stdout: mon pg warn max object skew = 0 2024-09-17T04:34:15.767 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.767 INFO:teuthology.orchestra.run.smithi037.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T04:34:15.767 INFO:teuthology.orchestra.run.smithi037.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T04:34:15.767 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.767 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default size = 2 2024-09-17T04:34:15.767 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.767 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow primary affinity = true 2024-09-17T04:34:15.767 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow pg remap = true 2024-09-17T04:34:15.767 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on legacy crush tunables = false 2024-09-17T04:34:15.767 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on crush straw calc version zero = false 2024-09-17T04:34:15.768 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on no sortbitwise = false 2024-09-17T04:34:15.768 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on osd down out interval zero = false 2024-09-17T04:34:15.768 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on too few osds = false 2024-09-17T04:34:15.768 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T04:34:15.768 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T04:34:15.768 INFO:teuthology.orchestra.run.smithi037.stdout: mon_allow_pool_size_one = true 2024-09-17T04:34:15.768 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.768 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T04:34:15.768 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.768 INFO:teuthology.orchestra.run.smithi037.stdout: osd default data pool replay window = 5 2024-09-17T04:34:15.768 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.768 INFO:teuthology.orchestra.run.smithi037.stdout: mon allow pool delete = true 2024-09-17T04:34:15.768 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.769 INFO:teuthology.orchestra.run.smithi037.stdout: mon cluster log file level = debug 2024-09-17T04:34:15.769 INFO:teuthology.orchestra.run.smithi037.stdout: debug asserts on shutdown = true 2024-09-17T04:34:15.769 INFO:teuthology.orchestra.run.smithi037.stdout: mon health detail to clog = false 2024-09-17T04:34:15.769 INFO:teuthology.orchestra.run.smithi037.stdout: mon host = "172.21.15.37,172.21.15.92,[v2:172.21.15.92:3301,v1:172.21.15.92:6790]" 2024-09-17T04:34:15.769 INFO:teuthology.orchestra.run.smithi037.stdout: fsid = a908a603-430c-4209-8294-0c3460b35f42 2024-09-17T04:34:15.769 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.769 INFO:teuthology.orchestra.run.smithi037.stdout:[osd] 2024-09-17T04:34:15.769 INFO:teuthology.orchestra.run.smithi037.stdout: osd journal size = 100 2024-09-17T04:34:15.769 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.769 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub load threshold = 5.0 2024-09-17T04:34:15.769 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub max interval = 600 2024-09-17T04:34:15.769 INFO:teuthology.orchestra.run.smithi037.stdout: osd mclock profile = high_recovery_ops 2024-09-17T04:34:15.769 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.769 INFO:teuthology.orchestra.run.smithi037.stdout: osd recover clone overlap = true 2024-09-17T04:34:15.770 INFO:teuthology.orchestra.run.smithi037.stdout: osd recovery max chunk = 1048576 2024-09-17T04:34:15.770 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.770 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug shutdown = true 2024-09-17T04:34:15.770 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug op order = true 2024-09-17T04:34:15.770 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug verify stray on activate = true 2024-09-17T04:34:15.770 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug trim objects = true 2024-09-17T04:34:15.770 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.770 INFO:teuthology.orchestra.run.smithi037.stdout: osd open classes on start = true 2024-09-17T04:34:15.770 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug pg log writeout = true 2024-09-17T04:34:15.770 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.770 INFO:teuthology.orchestra.run.smithi037.stdout: osd deep scrub update digest min age = 30 2024-09-17T04:34:15.770 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.770 INFO:teuthology.orchestra.run.smithi037.stdout: osd map max advance = 10 2024-09-17T04:34:15.770 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.771 INFO:teuthology.orchestra.run.smithi037.stdout: journal zero on create = true 2024-09-17T04:34:15.771 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.771 INFO:teuthology.orchestra.run.smithi037.stdout: filestore ondisk finisher threads = 3 2024-09-17T04:34:15.771 INFO:teuthology.orchestra.run.smithi037.stdout: filestore apply finisher threads = 3 2024-09-17T04:34:15.771 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.771 INFO:teuthology.orchestra.run.smithi037.stdout: bdev debug aio = true 2024-09-17T04:34:15.771 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug misdirected ops = true 2024-09-17T04:34:15.771 INFO:teuthology.orchestra.run.smithi037.stdout: bdev async discard = True 2024-09-17T04:34:15.771 INFO:teuthology.orchestra.run.smithi037.stdout: bdev enable discard = True 2024-09-17T04:34:15.771 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore allocator = bitmap 2024-09-17T04:34:15.771 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore block size = 96636764160 2024-09-17T04:34:15.771 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore fsck on mount = True 2024-09-17T04:34:15.771 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluefs = 1/20 2024-09-17T04:34:15.771 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluestore = 1/20 2024-09-17T04:34:15.772 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-09-17T04:34:15.772 INFO:teuthology.orchestra.run.smithi037.stdout: debug osd = 20 2024-09-17T04:34:15.772 INFO:teuthology.orchestra.run.smithi037.stdout: debug rocksdb = 4/10 2024-09-17T04:34:15.772 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T04:34:15.772 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd full ratio = 0.9 2024-09-17T04:34:15.772 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd nearfull ratio = 0.8 2024-09-17T04:34:15.772 INFO:teuthology.orchestra.run.smithi037.stdout: osd failsafe full ratio = 0.95 2024-09-17T04:34:15.772 INFO:teuthology.orchestra.run.smithi037.stdout: osd objectstore = bluestore 2024-09-17T04:34:15.772 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.772 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr] 2024-09-17T04:34:15.772 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-09-17T04:34:15.772 INFO:teuthology.orchestra.run.smithi037.stdout: debug mgr = 20 2024-09-17T04:34:15.772 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-09-17T04:34:15.772 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-09-17T04:34:15.773 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-09-17T04:34:15.773 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-09-17T04:34:15.773 INFO:teuthology.orchestra.run.smithi037.stdout: mgr/telemetry/nag = false 2024-09-17T04:34:15.773 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.773 INFO:teuthology.orchestra.run.smithi037.stdout:[mon] 2024-09-17T04:34:15.773 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-09-17T04:34:15.773 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-09-17T04:34:15.773 INFO:teuthology.orchestra.run.smithi037.stdout: debug paxos = 20 2024-09-17T04:34:15.773 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-09-17T04:34:15.773 INFO:teuthology.orchestra.run.smithi037.stdout: mon data avail warn = 5 2024-09-17T04:34:15.773 INFO:teuthology.orchestra.run.smithi037.stdout: mon mgr mkfs grace = 240 2024-09-17T04:34:15.773 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-09-17T04:34:15.773 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd reporter subtree level = osd 2024-09-17T04:34:15.774 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd prime pg temp = true 2024-09-17T04:34:15.774 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-09-17T04:34:15.774 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.774 INFO:teuthology.orchestra.run.smithi037.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T04:34:15.774 INFO:teuthology.orchestra.run.smithi037.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T04:34:15.774 INFO:teuthology.orchestra.run.smithi037.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T04:34:15.774 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.774 INFO:teuthology.orchestra.run.smithi037.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T04:34:15.774 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T04:34:15.774 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T04:34:15.774 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.774 INFO:teuthology.orchestra.run.smithi037.stdout: # 1m isn't quite enough 2024-09-17T04:34:15.774 INFO:teuthology.orchestra.run.smithi037.stdout: mon_down_mkfs_grace = 2m 2024-09-17T04:34:15.774 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.775 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_filestore_osds = false 2024-09-17T04:34:15.775 INFO:teuthology.orchestra.run.smithi037.stdout: mon down mkfs grace = 300 2024-09-17T04:34:15.775 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.775 INFO:teuthology.orchestra.run.smithi037.stdout:[client] 2024-09-17T04:34:15.775 INFO:teuthology.orchestra.run.smithi037.stdout: rgw cache enabled = true 2024-09-17T04:34:15.775 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable ops log = true 2024-09-17T04:34:15.775 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable usage log = true 2024-09-17T04:34:15.775 INFO:teuthology.orchestra.run.smithi037.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T04:34:15.775 INFO:teuthology.orchestra.run.smithi037.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T04:34:15.775 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = "" 2024-09-17T04:34:15.775 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.a] 2024-09-17T04:34:15.775 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.b] 2024-09-17T04:34:15.775 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.c] 2024-09-17T04:34:15.776 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.a] 2024-09-17T04:34:15.776 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.c] 2024-09-17T04:34:15.776 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.b] 2024-09-17T04:34:15.776 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.d] 2024-09-17T04:34:15.776 INFO:teuthology.orchestra.run.smithi037.stdout:[mds] 2024-09-17T04:34:15.776 INFO:teuthology.orchestra.run.smithi037.stdout: mds verify scatter = False 2024-09-17T04:34:15.776 INFO:teuthology.orchestra.run.smithi092.stdout:[global] 2024-09-17T04:34:15.776 INFO:teuthology.orchestra.run.smithi092.stdout: chdir = "" 2024-09-17T04:34:15.777 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T04:34:15.777 INFO:teuthology.orchestra.run.smithi092.stdout: auth supported = cephx 2024-09-17T04:34:15.777 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.777 INFO:teuthology.orchestra.run.smithi092.stdout: filestore xattr use omap = true 2024-09-17T04:34:15.777 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.777 INFO:teuthology.orchestra.run.smithi092.stdout: mon clock drift allowed = 1.000 2024-09-17T04:34:15.777 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.777 INFO:teuthology.orchestra.run.smithi092.stdout: osd crush chooseleaf type = 0 2024-09-17T04:34:15.777 INFO:teuthology.orchestra.run.smithi092.stdout: auth debug = true 2024-09-17T04:34:15.777 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.777 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on old message = true 2024-09-17T04:34:15.777 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on bug = true 2024-09-17T04:34:15.777 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.778 INFO:teuthology.orchestra.run.smithi092.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T04:34:15.778 INFO:teuthology.orchestra.run.smithi092.stdout: mon pg warn max object skew = 0 2024-09-17T04:34:15.778 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.778 INFO:teuthology.orchestra.run.smithi092.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T04:34:15.778 INFO:teuthology.orchestra.run.smithi092.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T04:34:15.778 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.778 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default size = 2 2024-09-17T04:34:15.778 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.778 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow primary affinity = true 2024-09-17T04:34:15.778 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow pg remap = true 2024-09-17T04:34:15.778 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on legacy crush tunables = false 2024-09-17T04:34:15.778 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on crush straw calc version zero = false 2024-09-17T04:34:15.778 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on no sortbitwise = false 2024-09-17T04:34:15.778 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on osd down out interval zero = false 2024-09-17T04:34:15.779 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on too few osds = false 2024-09-17T04:34:15.779 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T04:34:15.779 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T04:34:15.779 INFO:teuthology.orchestra.run.smithi092.stdout: mon_allow_pool_size_one = true 2024-09-17T04:34:15.779 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.779 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T04:34:15.779 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.779 INFO:teuthology.orchestra.run.smithi092.stdout: osd default data pool replay window = 5 2024-09-17T04:34:15.779 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.779 INFO:teuthology.orchestra.run.smithi092.stdout: mon allow pool delete = true 2024-09-17T04:34:15.779 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.779 INFO:teuthology.orchestra.run.smithi092.stdout: mon cluster log file level = debug 2024-09-17T04:34:15.779 INFO:teuthology.orchestra.run.smithi092.stdout: debug asserts on shutdown = true 2024-09-17T04:34:15.779 INFO:teuthology.orchestra.run.smithi092.stdout: mon health detail to clog = false 2024-09-17T04:34:15.780 INFO:teuthology.orchestra.run.smithi092.stdout: mon host = "172.21.15.37,172.21.15.92,[v2:172.21.15.92:3301,v1:172.21.15.92:6790]" 2024-09-17T04:34:15.780 INFO:teuthology.orchestra.run.smithi092.stdout: fsid = a908a603-430c-4209-8294-0c3460b35f42 2024-09-17T04:34:15.780 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.780 INFO:teuthology.orchestra.run.smithi092.stdout:[osd] 2024-09-17T04:34:15.780 INFO:teuthology.orchestra.run.smithi092.stdout: osd journal size = 100 2024-09-17T04:34:15.780 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.780 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub load threshold = 5.0 2024-09-17T04:34:15.780 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub max interval = 600 2024-09-17T04:34:15.780 INFO:teuthology.orchestra.run.smithi092.stdout: osd mclock profile = high_recovery_ops 2024-09-17T04:34:15.780 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.780 INFO:teuthology.orchestra.run.smithi092.stdout: osd recover clone overlap = true 2024-09-17T04:34:15.780 INFO:teuthology.orchestra.run.smithi092.stdout: osd recovery max chunk = 1048576 2024-09-17T04:34:15.780 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.780 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug shutdown = true 2024-09-17T04:34:15.781 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug op order = true 2024-09-17T04:34:15.781 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug verify stray on activate = true 2024-09-17T04:34:15.781 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug trim objects = true 2024-09-17T04:34:15.781 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.781 INFO:teuthology.orchestra.run.smithi092.stdout: osd open classes on start = true 2024-09-17T04:34:15.781 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug pg log writeout = true 2024-09-17T04:34:15.781 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.781 INFO:teuthology.orchestra.run.smithi092.stdout: osd deep scrub update digest min age = 30 2024-09-17T04:34:15.781 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.781 INFO:teuthology.orchestra.run.smithi092.stdout: osd map max advance = 10 2024-09-17T04:34:15.781 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.781 INFO:teuthology.orchestra.run.smithi092.stdout: journal zero on create = true 2024-09-17T04:34:15.781 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.781 INFO:teuthology.orchestra.run.smithi092.stdout: filestore ondisk finisher threads = 3 2024-09-17T04:34:15.781 INFO:teuthology.orchestra.run.smithi092.stdout: filestore apply finisher threads = 3 2024-09-17T04:34:15.782 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.782 INFO:teuthology.orchestra.run.smithi092.stdout: bdev debug aio = true 2024-09-17T04:34:15.782 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug misdirected ops = true 2024-09-17T04:34:15.782 INFO:teuthology.orchestra.run.smithi092.stdout: bdev async discard = True 2024-09-17T04:34:15.782 INFO:teuthology.orchestra.run.smithi092.stdout: bdev enable discard = True 2024-09-17T04:34:15.782 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore allocator = bitmap 2024-09-17T04:34:15.782 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore block size = 96636764160 2024-09-17T04:34:15.782 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore fsck on mount = True 2024-09-17T04:34:15.782 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluefs = 1/20 2024-09-17T04:34:15.782 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluestore = 1/20 2024-09-17T04:34:15.782 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-09-17T04:34:15.782 INFO:teuthology.orchestra.run.smithi092.stdout: debug osd = 20 2024-09-17T04:34:15.782 INFO:teuthology.orchestra.run.smithi092.stdout: debug rocksdb = 4/10 2024-09-17T04:34:15.782 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T04:34:15.783 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd full ratio = 0.9 2024-09-17T04:34:15.783 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd nearfull ratio = 0.8 2024-09-17T04:34:15.783 INFO:teuthology.orchestra.run.smithi092.stdout: osd failsafe full ratio = 0.95 2024-09-17T04:34:15.783 INFO:teuthology.orchestra.run.smithi092.stdout: osd objectstore = bluestore 2024-09-17T04:34:15.783 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.783 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr] 2024-09-17T04:34:15.783 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-09-17T04:34:15.783 INFO:teuthology.orchestra.run.smithi092.stdout: debug mgr = 20 2024-09-17T04:34:15.783 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2024-09-17T04:34:15.783 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2024-09-17T04:34:15.783 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2024-09-17T04:34:15.783 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2024-09-17T04:34:15.783 INFO:teuthology.orchestra.run.smithi092.stdout: mgr/telemetry/nag = false 2024-09-17T04:34:15.783 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.784 INFO:teuthology.orchestra.run.smithi092.stdout:[mon] 2024-09-17T04:34:15.784 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-09-17T04:34:15.784 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2024-09-17T04:34:15.784 INFO:teuthology.orchestra.run.smithi092.stdout: debug paxos = 20 2024-09-17T04:34:15.784 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2024-09-17T04:34:15.784 INFO:teuthology.orchestra.run.smithi092.stdout: mon data avail warn = 5 2024-09-17T04:34:15.784 INFO:teuthology.orchestra.run.smithi092.stdout: mon mgr mkfs grace = 240 2024-09-17T04:34:15.784 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2024-09-17T04:34:15.784 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd reporter subtree level = osd 2024-09-17T04:34:15.784 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd prime pg temp = true 2024-09-17T04:34:15.784 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2024-09-17T04:34:15.784 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.784 INFO:teuthology.orchestra.run.smithi092.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T04:34:15.784 INFO:teuthology.orchestra.run.smithi092.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T04:34:15.785 INFO:teuthology.orchestra.run.smithi092.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T04:34:15.785 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.785 INFO:teuthology.orchestra.run.smithi092.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T04:34:15.785 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T04:34:15.785 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T04:34:15.785 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.785 INFO:teuthology.orchestra.run.smithi092.stdout: # 1m isn't quite enough 2024-09-17T04:34:15.785 INFO:teuthology.orchestra.run.smithi092.stdout: mon_down_mkfs_grace = 2m 2024-09-17T04:34:15.785 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.785 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_filestore_osds = false 2024-09-17T04:34:15.785 INFO:teuthology.orchestra.run.smithi092.stdout: mon down mkfs grace = 300 2024-09-17T04:34:15.785 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.785 INFO:teuthology.orchestra.run.smithi092.stdout:[client] 2024-09-17T04:34:15.785 INFO:teuthology.orchestra.run.smithi092.stdout: rgw cache enabled = true 2024-09-17T04:34:15.785 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable ops log = true 2024-09-17T04:34:15.786 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable usage log = true 2024-09-17T04:34:15.786 INFO:teuthology.orchestra.run.smithi092.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T04:34:15.786 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T04:34:15.786 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = "" 2024-09-17T04:34:15.786 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.a] 2024-09-17T04:34:15.786 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.b] 2024-09-17T04:34:15.786 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.c] 2024-09-17T04:34:15.786 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.a] 2024-09-17T04:34:15.786 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.c] 2024-09-17T04:34:15.786 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.b] 2024-09-17T04:34:15.786 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.d] 2024-09-17T04:34:15.786 INFO:teuthology.orchestra.run.smithi092.stdout:[mds] 2024-09-17T04:34:15.786 INFO:teuthology.orchestra.run.smithi092.stdout: mds verify scatter = False 2024-09-17T04:34:15.788 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T04:34:15.831 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T04:34:15.874 INFO:teuthology.orchestra.run.smithi037.stdout:[global] 2024-09-17T04:34:15.874 INFO:teuthology.orchestra.run.smithi037.stdout: chdir = "" 2024-09-17T04:34:15.874 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T04:34:15.874 INFO:teuthology.orchestra.run.smithi037.stdout: auth supported = cephx 2024-09-17T04:34:15.874 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.874 INFO:teuthology.orchestra.run.smithi037.stdout: filestore xattr use omap = true 2024-09-17T04:34:15.874 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.874 INFO:teuthology.orchestra.run.smithi037.stdout: mon clock drift allowed = 1.000 2024-09-17T04:34:15.874 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.874 INFO:teuthology.orchestra.run.smithi037.stdout: osd crush chooseleaf type = 0 2024-09-17T04:34:15.874 INFO:teuthology.orchestra.run.smithi037.stdout: auth debug = true 2024-09-17T04:34:15.874 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.875 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on old message = true 2024-09-17T04:34:15.875 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on bug = true 2024-09-17T04:34:15.875 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.875 INFO:teuthology.orchestra.run.smithi037.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T04:34:15.875 INFO:teuthology.orchestra.run.smithi037.stdout: mon pg warn max object skew = 0 2024-09-17T04:34:15.875 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.875 INFO:teuthology.orchestra.run.smithi037.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T04:34:15.875 INFO:teuthology.orchestra.run.smithi037.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T04:34:15.875 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.875 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default size = 2 2024-09-17T04:34:15.875 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.875 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow primary affinity = true 2024-09-17T04:34:15.875 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow pg remap = true 2024-09-17T04:34:15.875 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on legacy crush tunables = false 2024-09-17T04:34:15.875 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on crush straw calc version zero = false 2024-09-17T04:34:15.876 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on no sortbitwise = false 2024-09-17T04:34:15.876 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on osd down out interval zero = false 2024-09-17T04:34:15.876 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on too few osds = false 2024-09-17T04:34:15.876 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T04:34:15.876 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T04:34:15.876 INFO:teuthology.orchestra.run.smithi037.stdout: mon_allow_pool_size_one = true 2024-09-17T04:34:15.876 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.876 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T04:34:15.876 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.876 INFO:teuthology.orchestra.run.smithi037.stdout: osd default data pool replay window = 5 2024-09-17T04:34:15.876 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.876 INFO:teuthology.orchestra.run.smithi037.stdout: mon allow pool delete = true 2024-09-17T04:34:15.876 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.877 INFO:teuthology.orchestra.run.smithi037.stdout: mon cluster log file level = debug 2024-09-17T04:34:15.877 INFO:teuthology.orchestra.run.smithi037.stdout: debug asserts on shutdown = true 2024-09-17T04:34:15.877 INFO:teuthology.orchestra.run.smithi037.stdout: mon health detail to clog = false 2024-09-17T04:34:15.877 INFO:teuthology.orchestra.run.smithi037.stdout: mon host = "172.21.15.37,172.21.15.92,[v2:172.21.15.92:3301,v1:172.21.15.92:6790]" 2024-09-17T04:34:15.877 INFO:teuthology.orchestra.run.smithi037.stdout: fsid = a908a603-430c-4209-8294-0c3460b35f42 2024-09-17T04:34:15.877 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.877 INFO:teuthology.orchestra.run.smithi037.stdout:[osd] 2024-09-17T04:34:15.877 INFO:teuthology.orchestra.run.smithi037.stdout: osd journal size = 100 2024-09-17T04:34:15.877 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.877 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub load threshold = 5.0 2024-09-17T04:34:15.877 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub max interval = 600 2024-09-17T04:34:15.877 INFO:teuthology.orchestra.run.smithi037.stdout: osd mclock profile = high_recovery_ops 2024-09-17T04:34:15.877 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.877 INFO:teuthology.orchestra.run.smithi037.stdout: osd recover clone overlap = true 2024-09-17T04:34:15.878 INFO:teuthology.orchestra.run.smithi037.stdout: osd recovery max chunk = 1048576 2024-09-17T04:34:15.878 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.878 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug shutdown = true 2024-09-17T04:34:15.878 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug op order = true 2024-09-17T04:34:15.878 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug verify stray on activate = true 2024-09-17T04:34:15.878 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug trim objects = true 2024-09-17T04:34:15.878 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.878 INFO:teuthology.orchestra.run.smithi037.stdout: osd open classes on start = true 2024-09-17T04:34:15.878 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug pg log writeout = true 2024-09-17T04:34:15.878 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.878 INFO:teuthology.orchestra.run.smithi037.stdout: osd deep scrub update digest min age = 30 2024-09-17T04:34:15.878 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.878 INFO:teuthology.orchestra.run.smithi037.stdout: osd map max advance = 10 2024-09-17T04:34:15.879 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.879 INFO:teuthology.orchestra.run.smithi037.stdout: journal zero on create = true 2024-09-17T04:34:15.879 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.879 INFO:teuthology.orchestra.run.smithi037.stdout: filestore ondisk finisher threads = 3 2024-09-17T04:34:15.879 INFO:teuthology.orchestra.run.smithi037.stdout: filestore apply finisher threads = 3 2024-09-17T04:34:15.879 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.879 INFO:teuthology.orchestra.run.smithi037.stdout: bdev debug aio = true 2024-09-17T04:34:15.879 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug misdirected ops = true 2024-09-17T04:34:15.879 INFO:teuthology.orchestra.run.smithi037.stdout: bdev async discard = True 2024-09-17T04:34:15.879 INFO:teuthology.orchestra.run.smithi037.stdout: bdev enable discard = True 2024-09-17T04:34:15.879 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore allocator = bitmap 2024-09-17T04:34:15.879 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore block size = 96636764160 2024-09-17T04:34:15.879 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore fsck on mount = True 2024-09-17T04:34:15.879 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluefs = 1/20 2024-09-17T04:34:15.879 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluestore = 1/20 2024-09-17T04:34:15.880 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-09-17T04:34:15.880 INFO:teuthology.orchestra.run.smithi037.stdout: debug osd = 20 2024-09-17T04:34:15.880 INFO:teuthology.orchestra.run.smithi037.stdout: debug rocksdb = 4/10 2024-09-17T04:34:15.880 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T04:34:15.880 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd full ratio = 0.9 2024-09-17T04:34:15.880 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd nearfull ratio = 0.8 2024-09-17T04:34:15.880 INFO:teuthology.orchestra.run.smithi037.stdout: osd failsafe full ratio = 0.95 2024-09-17T04:34:15.880 INFO:teuthology.orchestra.run.smithi037.stdout: osd objectstore = bluestore 2024-09-17T04:34:15.880 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.880 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr] 2024-09-17T04:34:15.880 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-09-17T04:34:15.880 INFO:teuthology.orchestra.run.smithi037.stdout: debug mgr = 20 2024-09-17T04:34:15.880 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-09-17T04:34:15.880 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-09-17T04:34:15.880 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-09-17T04:34:15.881 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-09-17T04:34:15.881 INFO:teuthology.orchestra.run.smithi037.stdout: mgr/telemetry/nag = false 2024-09-17T04:34:15.881 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.881 INFO:teuthology.orchestra.run.smithi037.stdout:[mon] 2024-09-17T04:34:15.881 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-09-17T04:34:15.881 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-09-17T04:34:15.881 INFO:teuthology.orchestra.run.smithi037.stdout: debug paxos = 20 2024-09-17T04:34:15.881 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-09-17T04:34:15.881 INFO:teuthology.orchestra.run.smithi037.stdout: mon data avail warn = 5 2024-09-17T04:34:15.881 INFO:teuthology.orchestra.run.smithi037.stdout: mon mgr mkfs grace = 240 2024-09-17T04:34:15.881 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-09-17T04:34:15.881 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd reporter subtree level = osd 2024-09-17T04:34:15.881 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd prime pg temp = true 2024-09-17T04:34:15.881 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-09-17T04:34:15.881 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.881 INFO:teuthology.orchestra.run.smithi037.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T04:34:15.882 INFO:teuthology.orchestra.run.smithi037.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T04:34:15.882 INFO:teuthology.orchestra.run.smithi037.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T04:34:15.882 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.882 INFO:teuthology.orchestra.run.smithi037.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T04:34:15.882 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T04:34:15.882 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T04:34:15.882 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.882 INFO:teuthology.orchestra.run.smithi037.stdout: # 1m isn't quite enough 2024-09-17T04:34:15.882 INFO:teuthology.orchestra.run.smithi037.stdout: mon_down_mkfs_grace = 2m 2024-09-17T04:34:15.882 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.882 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_filestore_osds = false 2024-09-17T04:34:15.882 INFO:teuthology.orchestra.run.smithi037.stdout: mon down mkfs grace = 300 2024-09-17T04:34:15.882 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:15.882 INFO:teuthology.orchestra.run.smithi037.stdout:[client] 2024-09-17T04:34:15.882 INFO:teuthology.orchestra.run.smithi037.stdout: rgw cache enabled = true 2024-09-17T04:34:15.883 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable ops log = true 2024-09-17T04:34:15.883 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable usage log = true 2024-09-17T04:34:15.883 INFO:teuthology.orchestra.run.smithi037.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T04:34:15.883 INFO:teuthology.orchestra.run.smithi037.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T04:34:15.883 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = "" 2024-09-17T04:34:15.883 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.a] 2024-09-17T04:34:15.883 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.b] 2024-09-17T04:34:15.883 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.c] 2024-09-17T04:34:15.883 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.a] 2024-09-17T04:34:15.883 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.c] 2024-09-17T04:34:15.883 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.b] 2024-09-17T04:34:15.883 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.d] 2024-09-17T04:34:15.883 INFO:teuthology.orchestra.run.smithi037.stdout:[mds] 2024-09-17T04:34:15.883 INFO:teuthology.orchestra.run.smithi037.stdout: mds verify scatter = False 2024-09-17T04:34:15.883 INFO:teuthology.orchestra.run.smithi037.stdout: mds debug scatterstat = False 2024-09-17T04:34:15.885 INFO:teuthology.orchestra.run.smithi092.stdout:[global] 2024-09-17T04:34:15.885 INFO:teuthology.orchestra.run.smithi092.stdout: chdir = "" 2024-09-17T04:34:15.885 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T04:34:15.885 INFO:teuthology.orchestra.run.smithi092.stdout: auth supported = cephx 2024-09-17T04:34:15.885 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.885 INFO:teuthology.orchestra.run.smithi092.stdout: filestore xattr use omap = true 2024-09-17T04:34:15.885 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.885 INFO:teuthology.orchestra.run.smithi092.stdout: mon clock drift allowed = 1.000 2024-09-17T04:34:15.885 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.885 INFO:teuthology.orchestra.run.smithi092.stdout: osd crush chooseleaf type = 0 2024-09-17T04:34:15.885 INFO:teuthology.orchestra.run.smithi092.stdout: auth debug = true 2024-09-17T04:34:15.885 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.886 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on old message = true 2024-09-17T04:34:15.886 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on bug = true 2024-09-17T04:34:15.886 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.886 INFO:teuthology.orchestra.run.smithi092.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T04:34:15.886 INFO:teuthology.orchestra.run.smithi092.stdout: mon pg warn max object skew = 0 2024-09-17T04:34:15.886 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.886 INFO:teuthology.orchestra.run.smithi092.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T04:34:15.886 INFO:teuthology.orchestra.run.smithi092.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T04:34:15.886 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.886 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default size = 2 2024-09-17T04:34:15.886 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.886 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow primary affinity = true 2024-09-17T04:34:15.886 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow pg remap = true 2024-09-17T04:34:15.887 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on legacy crush tunables = false 2024-09-17T04:34:15.887 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on crush straw calc version zero = false 2024-09-17T04:34:15.887 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on no sortbitwise = false 2024-09-17T04:34:15.887 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on osd down out interval zero = false 2024-09-17T04:34:15.887 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on too few osds = false 2024-09-17T04:34:15.887 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T04:34:15.887 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T04:34:15.887 INFO:teuthology.orchestra.run.smithi092.stdout: mon_allow_pool_size_one = true 2024-09-17T04:34:15.887 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.887 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T04:34:15.887 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.887 INFO:teuthology.orchestra.run.smithi092.stdout: osd default data pool replay window = 5 2024-09-17T04:34:15.887 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.887 INFO:teuthology.orchestra.run.smithi092.stdout: mon allow pool delete = true 2024-09-17T04:34:15.887 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.888 INFO:teuthology.orchestra.run.smithi092.stdout: mon cluster log file level = debug 2024-09-17T04:34:15.888 INFO:teuthology.orchestra.run.smithi092.stdout: debug asserts on shutdown = true 2024-09-17T04:34:15.888 INFO:teuthology.orchestra.run.smithi092.stdout: mon health detail to clog = false 2024-09-17T04:34:15.888 INFO:teuthology.orchestra.run.smithi092.stdout: mon host = "172.21.15.37,172.21.15.92,[v2:172.21.15.92:3301,v1:172.21.15.92:6790]" 2024-09-17T04:34:15.888 INFO:teuthology.orchestra.run.smithi092.stdout: fsid = a908a603-430c-4209-8294-0c3460b35f42 2024-09-17T04:34:15.888 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.888 INFO:teuthology.orchestra.run.smithi092.stdout:[osd] 2024-09-17T04:34:15.888 INFO:teuthology.orchestra.run.smithi092.stdout: osd journal size = 100 2024-09-17T04:34:15.888 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.888 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub load threshold = 5.0 2024-09-17T04:34:15.888 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub max interval = 600 2024-09-17T04:34:15.888 INFO:teuthology.orchestra.run.smithi092.stdout: osd mclock profile = high_recovery_ops 2024-09-17T04:34:15.888 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.888 INFO:teuthology.orchestra.run.smithi092.stdout: osd recover clone overlap = true 2024-09-17T04:34:15.889 INFO:teuthology.orchestra.run.smithi092.stdout: osd recovery max chunk = 1048576 2024-09-17T04:34:15.889 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.889 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug shutdown = true 2024-09-17T04:34:15.889 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug op order = true 2024-09-17T04:34:15.889 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug verify stray on activate = true 2024-09-17T04:34:15.889 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug trim objects = true 2024-09-17T04:34:15.889 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.889 INFO:teuthology.orchestra.run.smithi092.stdout: osd open classes on start = true 2024-09-17T04:34:15.889 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug pg log writeout = true 2024-09-17T04:34:15.889 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.889 INFO:teuthology.orchestra.run.smithi092.stdout: osd deep scrub update digest min age = 30 2024-09-17T04:34:15.889 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.889 INFO:teuthology.orchestra.run.smithi092.stdout: osd map max advance = 10 2024-09-17T04:34:15.890 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.890 INFO:teuthology.orchestra.run.smithi092.stdout: journal zero on create = true 2024-09-17T04:34:15.890 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.890 INFO:teuthology.orchestra.run.smithi092.stdout: filestore ondisk finisher threads = 3 2024-09-17T04:34:15.890 INFO:teuthology.orchestra.run.smithi092.stdout: filestore apply finisher threads = 3 2024-09-17T04:34:15.890 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.890 INFO:teuthology.orchestra.run.smithi092.stdout: bdev debug aio = true 2024-09-17T04:34:15.890 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug misdirected ops = true 2024-09-17T04:34:15.890 INFO:teuthology.orchestra.run.smithi092.stdout: bdev async discard = True 2024-09-17T04:34:15.890 INFO:teuthology.orchestra.run.smithi092.stdout: bdev enable discard = True 2024-09-17T04:34:15.890 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore allocator = bitmap 2024-09-17T04:34:15.890 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore block size = 96636764160 2024-09-17T04:34:15.890 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore fsck on mount = True 2024-09-17T04:34:15.890 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluefs = 1/20 2024-09-17T04:34:15.890 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluestore = 1/20 2024-09-17T04:34:15.891 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-09-17T04:34:15.891 INFO:teuthology.orchestra.run.smithi092.stdout: debug osd = 20 2024-09-17T04:34:15.891 INFO:teuthology.orchestra.run.smithi092.stdout: debug rocksdb = 4/10 2024-09-17T04:34:15.891 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T04:34:15.891 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd full ratio = 0.9 2024-09-17T04:34:15.891 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd nearfull ratio = 0.8 2024-09-17T04:34:15.891 INFO:teuthology.orchestra.run.smithi092.stdout: osd failsafe full ratio = 0.95 2024-09-17T04:34:15.891 INFO:teuthology.orchestra.run.smithi092.stdout: osd objectstore = bluestore 2024-09-17T04:34:15.891 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.891 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr] 2024-09-17T04:34:15.891 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-09-17T04:34:15.891 INFO:teuthology.orchestra.run.smithi092.stdout: debug mgr = 20 2024-09-17T04:34:15.891 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2024-09-17T04:34:15.891 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2024-09-17T04:34:15.891 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2024-09-17T04:34:15.892 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2024-09-17T04:34:15.892 INFO:teuthology.orchestra.run.smithi092.stdout: mgr/telemetry/nag = false 2024-09-17T04:34:15.892 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.892 INFO:teuthology.orchestra.run.smithi092.stdout:[mon] 2024-09-17T04:34:15.892 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-09-17T04:34:15.892 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2024-09-17T04:34:15.892 INFO:teuthology.orchestra.run.smithi092.stdout: debug paxos = 20 2024-09-17T04:34:15.892 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2024-09-17T04:34:15.892 INFO:teuthology.orchestra.run.smithi092.stdout: mon data avail warn = 5 2024-09-17T04:34:15.892 INFO:teuthology.orchestra.run.smithi092.stdout: mon mgr mkfs grace = 240 2024-09-17T04:34:15.892 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2024-09-17T04:34:15.892 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd reporter subtree level = osd 2024-09-17T04:34:15.892 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd prime pg temp = true 2024-09-17T04:34:15.893 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2024-09-17T04:34:15.893 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.893 INFO:teuthology.orchestra.run.smithi092.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T04:34:15.893 INFO:teuthology.orchestra.run.smithi092.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T04:34:15.893 INFO:teuthology.orchestra.run.smithi092.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T04:34:15.893 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.893 INFO:teuthology.orchestra.run.smithi092.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T04:34:15.893 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T04:34:15.893 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T04:34:15.893 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.893 INFO:teuthology.orchestra.run.smithi092.stdout: # 1m isn't quite enough 2024-09-17T04:34:15.893 INFO:teuthology.orchestra.run.smithi092.stdout: mon_down_mkfs_grace = 2m 2024-09-17T04:34:15.893 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.893 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_filestore_osds = false 2024-09-17T04:34:15.894 INFO:teuthology.orchestra.run.smithi092.stdout: mon down mkfs grace = 300 2024-09-17T04:34:15.894 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-17T04:34:15.894 INFO:teuthology.orchestra.run.smithi092.stdout:[client] 2024-09-17T04:34:15.894 INFO:teuthology.orchestra.run.smithi092.stdout: rgw cache enabled = true 2024-09-17T04:34:15.894 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable ops log = true 2024-09-17T04:34:15.894 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable usage log = true 2024-09-17T04:34:15.894 INFO:teuthology.orchestra.run.smithi092.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T04:34:15.894 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T04:34:15.894 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = "" 2024-09-17T04:34:15.894 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.a] 2024-09-17T04:34:15.894 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.b] 2024-09-17T04:34:15.894 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.c] 2024-09-17T04:34:15.894 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.a] 2024-09-17T04:34:15.894 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.c] 2024-09-17T04:34:15.894 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.b] 2024-09-17T04:34:15.895 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.d] 2024-09-17T04:34:15.895 INFO:teuthology.orchestra.run.smithi092.stdout:[mds] 2024-09-17T04:34:15.895 INFO:teuthology.orchestra.run.smithi092.stdout: mds verify scatter = False 2024-09-17T04:34:15.895 INFO:teuthology.orchestra.run.smithi092.stdout: mds debug scatterstat = False 2024-09-17T04:34:15.899 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T04:34:15.899 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T04:34:15.899 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T04:34:15.899 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:34:15.899 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T04:34:15.899 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T04:34:15.900 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T04:34:15.900 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:34:15.900 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T04:34:15.900 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T04:34:15.900 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T04:34:15.900 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:34:15.900 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T04:34:15.900 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T04:34:15.900 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T04:34:15.901 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:34:15.947 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T04:34:15.948 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T04:34:15.949 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T04:34:15.950 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T04:34:15.951 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T04:34:15.951 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T04:34:15.952 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T04:34:15.952 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T04:34:15.991 INFO:tasks.ceph.mds.c:Started 2024-09-17T04:34:15.992 DEBUG:teuthology.parallel:result is None 2024-09-17T04:34:15.993 INFO:tasks.ceph.mds.a:Started 2024-09-17T04:34:15.993 DEBUG:teuthology.parallel:result is None 2024-09-17T04:34:15.994 INFO:tasks.ceph.mds.b:Started 2024-09-17T04:34:15.994 DEBUG:teuthology.parallel:result is None 2024-09-17T04:34:15.996 INFO:tasks.ceph.mds.d:Started 2024-09-17T04:34:15.997 DEBUG:teuthology.parallel:result is None 2024-09-17T04:34:15.997 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph purge cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T04:34:16.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.052+0000 7fc4e660bf40 1 Processor -- start 2024-09-17T04:34:16.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.052+0000 7fc4e660bf40 1 -- start start 2024-09-17T04:34:16.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.053+0000 7fc4e660bf40 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5630f0710580 0x5630f07d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:16.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.053+0000 7fc4e660bf40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630f0713ea0 0x5630f07586d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:16.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.053+0000 7fc4e660bf40 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630f07e7320 0x5630f07cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:16.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.053+0000 7fc4e660bf40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5630f0661d60 con 0x5630f0713ea0 2024-09-17T04:34:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.053+0000 7fc4e660bf40 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5630f06c0430 con 0x5630f07e7320 2024-09-17T04:34:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.053+0000 7fc4e660bf40 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x5630f0712150 con 0x5630f0710580 2024-09-17T04:34:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.053+0000 7fc4dec37640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630f0713ea0 0x5630f07586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.053+0000 7fc4dec37640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630f0713ea0 0x5630f07586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38350/0 (socket says 172.21.15.37:38350) 2024-09-17T04:34:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.053+0000 7fc4dec37640 1 -- 172.21.15.37:0/1568668684 learned_addr learned my addr 172.21.15.37:0/1568668684 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.053+0000 7fc4dfc39640 1 --2- 172.21.15.37:0/1568668684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630f07e7320 0x5630f07cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.054+0000 7fc4df438640 1 --2- 172.21.15.37:0/1568668684 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5630f0710580 0x5630f07d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.054+0000 7fc4dec37640 1 -- 172.21.15.37:0/1568668684 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5630f0710580 msgr2=0x5630f07d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.054+0000 7fc4dec37640 1 --2- 172.21.15.37:0/1568668684 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5630f0710580 0x5630f07d0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.054+0000 7fc4dec37640 1 -- 172.21.15.37:0/1568668684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630f07e7320 msgr2=0x5630f07cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.054+0000 7fc4dec37640 1 --2- 172.21.15.37:0/1568668684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630f07e7320 0x5630f07cf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:16.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.054+0000 7fc4dec37640 1 -- 172.21.15.37:0/1568668684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630f07e6700 con 0x5630f0713ea0 2024-09-17T04:34:16.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.054+0000 7fc4dec37640 1 --2- 172.21.15.37:0/1568668684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630f0713ea0 0x5630f07586d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4025de0 tx=0x7fc4d4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7106c2a2218ee502 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:16.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.054+0000 7fc4ddc35640 1 -- 172.21.15.37:0/1568668684 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d404dcd0 con 0x5630f0713ea0 2024-09-17T04:34:16.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.054+0000 7fc4ddc35640 1 -- 172.21.15.37:0/1568668684 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4d4056470 con 0x5630f0713ea0 2024-09-17T04:34:16.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.054+0000 7fc4ddc35640 1 -- 172.21.15.37:0/1568668684 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d40568a0 con 0x5630f0713ea0 2024-09-17T04:34:16.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.055+0000 7fc4e660bf40 1 -- 172.21.15.37:0/1568668684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630f0713ea0 msgr2=0x5630f07586d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:16.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.055+0000 7fc4e660bf40 1 --2- 172.21.15.37:0/1568668684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630f0713ea0 0x5630f07586d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4025de0 tx=0x7fc4d4002fb0 comp rx=0 tx=0).stop 2024-09-17T04:34:16.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.055+0000 7fc4e660bf40 1 -- 172.21.15.37:0/1568668684 shutdown_connections 2024-09-17T04:34:16.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.055+0000 7fc4e660bf40 1 --2- 172.21.15.37:0/1568668684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630f07e7320 0x5630f07cf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:16.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.055+0000 7fc4e660bf40 1 --2- 172.21.15.37:0/1568668684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630f0713ea0 0x5630f07586d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:16.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.055+0000 7fc4e660bf40 1 --2- 172.21.15.37:0/1568668684 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5630f0710580 0x5630f07d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:16.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.055+0000 7fc4e660bf40 1 -- 172.21.15.37:0/1568668684 >> 172.21.15.37:0/1568668684 conn(0x5630f0632ac0 msgr2=0x5630f0759ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:16.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.055+0000 7fc4e660bf40 1 -- 172.21.15.37:0/1568668684 shutdown_connections 2024-09-17T04:34:16.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.055+0000 7fc4e660bf40 1 -- 172.21.15.37:0/1568668684 wait complete. 2024-09-17T04:34:16.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.056+0000 7fc4e660bf40 1 Processor -- start 2024-09-17T04:34:16.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.056+0000 7fc4e660bf40 1 -- start start 2024-09-17T04:34:16.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.058+0000 7fc4e660bf40 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5630f0710580 0x5630f07ce250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:16.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.058+0000 7fc4e660bf40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630f0713ea0 0x5630f07568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:16.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.058+0000 7fc4e660bf40 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630f07e7320 0x5630f0883db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:16.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.058+0000 7fc4e660bf40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5630f07e6970 con 0x5630f0713ea0 2024-09-17T04:34:16.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.058+0000 7fc4e660bf40 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5630f073c000 con 0x5630f07e7320 2024-09-17T04:34:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.058+0000 7fc4e660bf40 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x5630f07e9db0 con 0x5630f0710580 2024-09-17T04:34:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.058+0000 7fc4dfc39640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630f07e7320 0x5630f0883db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.058+0000 7fc4df438640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5630f0710580 0x5630f07ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.058+0000 7fc4dec37640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630f0713ea0 0x5630f07568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.058+0000 7fc4df438640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5630f0710580 0x5630f07ce250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.37:36980/0 (socket says 172.21.15.37:36980) 2024-09-17T04:34:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.058+0000 7fc4dfc39640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630f07e7320 0x5630f0883db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.37:38714/0 (socket says 172.21.15.37:38714) 2024-09-17T04:34:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.058+0000 7fc4df438640 1 -- 172.21.15.37:0/988470404 learned_addr learned my addr 172.21.15.37:0/988470404 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.059+0000 7fc4df438640 1 -- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630f07e7320 msgr2=0x5630f0883db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.059+0000 7fc4df438640 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630f07e7320 0x5630f0883db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.059+0000 7fc4df438640 1 -- 172.21.15.37:0/988470404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630f0713ea0 msgr2=0x5630f07568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.059+0000 7fc4df438640 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5630f0713ea0 0x5630f07568a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.059+0000 7fc4df438640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630f0884b80 con 0x5630f0710580 2024-09-17T04:34:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.059+0000 7fc4dfc39640 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5630f07e7320 0x5630f0883db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:34:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.059+0000 7fc4df438640 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5630f0710580 0x5630f07ce250 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0002e80 tx=0x7fc4d000c7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:16.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.059+0000 7fc4dcc33640 1 -- 172.21.15.37:0/988470404 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d000f810 con 0x5630f0710580 2024-09-17T04:34:16.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.059+0000 7fc4dcc33640 1 -- 172.21.15.37:0/988470404 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4d0013070 con 0x5630f0710580 2024-09-17T04:34:16.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.059+0000 7fc4dcc33640 1 -- 172.21.15.37:0/988470404 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d000fd20 con 0x5630f0710580 2024-09-17T04:34:16.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.059+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x5630f07d03b0 con 0x5630f0710580 2024-09-17T04:34:16.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.060+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) -- 0x5630f0740860 con 0x5630f0710580 2024-09-17T04:34:16.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.062+0000 7fc4dcc33640 1 -- 172.21.15.37:0/988470404 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc4d0018470 con 0x5630f0710580 2024-09-17T04:34:16.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.063+0000 7fc4dcc33640 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc4cc077570 0x7fc4cc079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:16.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.063+0000 7fc4dcc33640 1 -- 172.21.15.37:0/988470404 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7fc4d009a280 con 0x5630f0710580 2024-09-17T04:34:16.064 INFO:teuthology.orchestra.run.smithi037.stdout:Warning: using slow linear search 2024-09-17T04:34:16.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.063+0000 7fc4dec37640 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc4cc077570 0x7fc4cc079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:16.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.063+0000 7fc4e660bf40 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x5630f088beb0 0x5630f088e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:16.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.063+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:1 4.0 4:00000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x5630f088beb0 2024-09-17T04:34:16.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.064+0000 7fc4dfc39640 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x5630f088beb0 0x5630f088e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:16.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.064+0000 7fc4dec37640 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc4cc077570 0x7fc4cc079a80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4025de0 tx=0x7fc4d4064040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:16.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.064+0000 7fc4dfc39640 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x5630f088beb0 0x5630f088e2d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:16.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.068+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.3 v2:172.21.15.37:6800/809975826 1 ==== osd_op_reply(1 [pgnls start_epoch 34 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc4d8003040 con 0x5630f088beb0 2024-09-17T04:34:16.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.068+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:2 4.10 4:08000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x5630f088beb0 2024-09-17T04:34:16.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.068+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.3 v2:172.21.15.37:6800/809975826 2 ==== osd_op_reply(2 [pgnls start_epoch 34 out=177b] v33'10 uv10 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x7fc4d8003040 con 0x5630f088beb0 2024-09-17T04:34:16.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.068+0000 7fc4dfc39640 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x7fc4d800e8d0 0x7fc4d800a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:16.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.068+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:3 4.8 4:10000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x7fc4d800e8d0 2024-09-17T04:34:16.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.068+0000 7fc4df438640 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x7fc4d800e8d0 0x7fc4d800a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:16.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.069+0000 7fc4df438640 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x7fc4d800e8d0 0x7fc4d800a2c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:16.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.072+0000 7fc4df438640 1 -- 172.21.15.37:0/988470404 <== osd.5 v2:172.21.15.92:6826/248263832 1 ==== osd_op_reply(3 [pgnls start_epoch 34 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc4d0009070 con 0x7fc4d800e8d0 2024-09-17T04:34:16.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.072+0000 7fc4df438640 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x7fc4d00622f0 0x7fc4d0060d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:16.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.072+0000 7fc4df438640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:4 4.18 4:18000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x7fc4d00622f0 2024-09-17T04:34:16.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.072+0000 7fc4dec37640 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x7fc4d00622f0 0x7fc4d0060d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:16.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.072+0000 7fc4dec37640 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x7fc4d00622f0 0x7fc4d0060d60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:16.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.073+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 <== osd.4 v2:172.21.15.92:6818/2509785752 1 ==== osd_op_reply(4 [pgnls start_epoch 34 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc4d406a040 con 0x7fc4d00622f0 2024-09-17T04:34:16.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.073+0000 7fc4dec37640 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x7fc4d4022da0 0x7fc4d4056470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:16.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.073+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:5 4.4 4:20000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x7fc4d4022da0 2024-09-17T04:34:16.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.073+0000 7fc4dfc39640 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x7fc4d4022da0 0x7fc4d4056470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:16.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.073+0000 7fc4dfc39640 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x7fc4d4022da0 0x7fc4d4056470 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:16.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.074+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.1 v2:172.21.15.37:6824/2657955935 1 ==== osd_op_reply(5 [pgnls start_epoch 34 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc4d8003040 con 0x7fc4d4022da0 2024-09-17T04:34:16.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.074+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:6 4.14 4:28000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x5630f088beb0 2024-09-17T04:34:16.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.074+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.3 v2:172.21.15.37:6800/809975826 3 ==== osd_op_reply(6 [pgnls start_epoch 34 out=149b] v33'19 uv19 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x7fc4d8003040 con 0x5630f088beb0 2024-09-17T04:34:16.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.074+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:7 4.c 4:30000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x7fc4d00622f0 2024-09-17T04:34:16.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.075+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 <== osd.4 v2:172.21.15.92:6818/2509785752 2 ==== osd_op_reply(7 [pgnls start_epoch 34 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc4d406a040 con 0x7fc4d00622f0 2024-09-17T04:34:16.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.075+0000 7fc4dec37640 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x7fc4d401f720 0x7fc4d4062410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:16.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.075+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:8 4.1c 4:38000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x7fc4d401f720 2024-09-17T04:34:16.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.075+0000 7fc4df438640 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x7fc4d401f720 0x7fc4d4062410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:16.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.075+0000 7fc4df438640 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x7fc4d401f720 0x7fc4d4062410 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:16.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.076+0000 7fc4df438640 1 -- 172.21.15.37:0/988470404 <== osd.2 v2:172.21.15.37:6816/579720176 1 ==== osd_op_reply(8 [pgnls start_epoch 34 out=73b] v32'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7fc4d0009070 con 0x7fc4d401f720 2024-09-17T04:34:16.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.076+0000 7fc4df438640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:9 4.2 4:40000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x7fc4d4022da0 2024-09-17T04:34:16.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.076+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.1 v2:172.21.15.37:6824/2657955935 2 ==== osd_op_reply(9 [pgnls start_epoch 34 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc4d8003040 con 0x7fc4d4022da0 2024-09-17T04:34:16.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.076+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:10 4.12 4:48000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x7fc4d4022da0 2024-09-17T04:34:16.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.076+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.1 v2:172.21.15.37:6824/2657955935 3 ==== osd_op_reply(10 [pgnls start_epoch 34 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc4d8003040 con 0x7fc4d4022da0 2024-09-17T04:34:16.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.076+0000 7fc4dfc39640 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] conn(0x7fc4d8021940 0x7fc4d80203c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:16.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.076+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] -- osd_op(unknown.0.0:11 4.a 4:50000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x7fc4d8021940 2024-09-17T04:34:16.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.077+0000 7fc4dec37640 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] conn(0x7fc4d8021940 0x7fc4d80203c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:16.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.077+0000 7fc4dec37640 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] conn(0x7fc4d8021940 0x7fc4d80203c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:16.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.080+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 <== osd.6 v2:172.21.15.92:6810/4078365985 1 ==== osd_op_reply(11 [pgnls start_epoch 34 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc4d406a040 con 0x7fc4d8021940 2024-09-17T04:34:16.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.080+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:12 4.1a 4:58000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x7fc4d00622f0 2024-09-17T04:34:16.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.080+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 <== osd.4 v2:172.21.15.92:6818/2509785752 3 ==== osd_op_reply(12 [pgnls start_epoch 34 out=49b] v32'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc4d406a040 con 0x7fc4d00622f0 2024-09-17T04:34:16.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.080+0000 7fc4dec37640 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x7fc4d400c2e0 0x7fc4d407e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:16.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.080+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:13 4.6 4:60000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x7fc4d400c2e0 2024-09-17T04:34:16.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.080+0000 7fc4dfc39640 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x7fc4d400c2e0 0x7fc4d407e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:16.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.081+0000 7fc4dfc39640 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x7fc4d400c2e0 0x7fc4d407e920 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:16.082 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-09-17T04:34:16.083 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-09-17T04:34:16.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.084+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.0 v2:172.21.15.37:6808/2283587593 1 ==== osd_op_reply(13 [pgnls start_epoch 34 out=77b] v33'4 uv4 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x7fc4d8003040 con 0x7fc4d400c2e0 2024-09-17T04:34:16.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.084+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:14 4.16 4:68000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x7fc4d400c2e0 2024-09-17T04:34:16.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.084+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.0 v2:172.21.15.37:6808/2283587593 2 ==== osd_op_reply(14 [pgnls start_epoch 34 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc4d8003040 con 0x7fc4d400c2e0 2024-09-17T04:34:16.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.084+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:15 4.e 4:70000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x7fc4d00622f0 2024-09-17T04:34:16.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.085+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 <== osd.4 v2:172.21.15.92:6818/2509785752 4 ==== osd_op_reply(15 [pgnls start_epoch 34 out=73b] v32'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7fc4d406a040 con 0x7fc4d00622f0 2024-09-17T04:34:16.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.085+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:16 4.1e 4:78000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x7fc4d400c2e0 2024-09-17T04:34:16.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.085+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.0 v2:172.21.15.37:6808/2283587593 3 ==== osd_op_reply(16 [pgnls start_epoch 34 out=49b] v32'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc4d8003040 con 0x7fc4d400c2e0 2024-09-17T04:34:16.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.085+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:17 4.1 4:80000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x7fc4d00622f0 2024-09-17T04:34:16.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.085+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 <== osd.4 v2:172.21.15.92:6818/2509785752 5 ==== osd_op_reply(17 [pgnls start_epoch 34 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc4d406a040 con 0x7fc4d00622f0 2024-09-17T04:34:16.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.085+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:18 4.11 4:88000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x5630f088beb0 2024-09-17T04:34:16.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.086+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.3 v2:172.21.15.37:6800/809975826 4 ==== osd_op_reply(18 [pgnls start_epoch 34 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc4d8003040 con 0x5630f088beb0 2024-09-17T04:34:16.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.086+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:19 4.9 4:90000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x7fc4d00622f0 2024-09-17T04:34:16.087 INFO:tasks.ceph.mds.d.smithi092.stdout:starting mds.d at 2024-09-17T04:34:16.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.086+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 <== osd.4 v2:172.21.15.92:6818/2509785752 6 ==== osd_op_reply(19 [pgnls start_epoch 34 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc4d406a040 con 0x7fc4d00622f0 2024-09-17T04:34:16.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.086+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:20 4.19 4:98000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x5630f088beb0 2024-09-17T04:34:16.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.087+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.3 v2:172.21.15.37:6800/809975826 5 ==== osd_op_reply(20 [pgnls start_epoch 34 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc4d8003040 con 0x5630f088beb0 2024-09-17T04:34:16.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.087+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] -- osd_op(unknown.0.0:21 4.5 4:a0000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x7fc4d8021940 2024-09-17T04:34:16.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.087+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 <== osd.6 v2:172.21.15.92:6810/4078365985 2 ==== osd_op_reply(21 [pgnls start_epoch 34 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc4d406a040 con 0x7fc4d8021940 2024-09-17T04:34:16.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.087+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:22 4.15 4:a8000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x7fc4d800e8d0 2024-09-17T04:34:16.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.088+0000 7fc4df438640 1 -- 172.21.15.37:0/988470404 <== osd.5 v2:172.21.15.92:6826/248263832 2 ==== osd_op_reply(22 [pgnls start_epoch 34 out=97b] v32'3 uv3 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x7fc4d0009070 con 0x7fc4d800e8d0 2024-09-17T04:34:16.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.088+0000 7fc4df438640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:23 4.d 4:b0000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x7fc4d00622f0 2024-09-17T04:34:16.089 INFO:tasks.ceph.mds.b.smithi092.stdout:starting mds.b at 2024-09-17T04:34:16.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.088+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 <== osd.4 v2:172.21.15.92:6818/2509785752 7 ==== osd_op_reply(23 [pgnls start_epoch 34 out=74b] v32'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fc4d406a040 con 0x7fc4d00622f0 2024-09-17T04:34:16.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.088+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:24 4.1d 4:b8000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x5630f088beb0 2024-09-17T04:34:16.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.088+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.3 v2:172.21.15.37:6800/809975826 6 ==== osd_op_reply(24 [pgnls start_epoch 34 out=73b] v32'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7fc4d8003040 con 0x5630f088beb0 2024-09-17T04:34:16.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.088+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:25 4.3 4:c0000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x7fc4d400c2e0 2024-09-17T04:34:16.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.089+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.0 v2:172.21.15.37:6808/2283587593 4 ==== osd_op_reply(25 [pgnls start_epoch 34 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc4d8003040 con 0x7fc4d400c2e0 2024-09-17T04:34:16.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.089+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:26 4.13 4:c8000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x7fc4d00622f0 2024-09-17T04:34:16.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.089+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 <== osd.4 v2:172.21.15.92:6818/2509785752 8 ==== osd_op_reply(26 [pgnls start_epoch 34 out=98b] v33'3 uv3 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x7fc4d406a040 con 0x7fc4d00622f0 2024-09-17T04:34:16.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.089+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:27 4.b 4:d0000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x7fc4d400c2e0 2024-09-17T04:34:16.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.090+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.0 v2:172.21.15.37:6808/2283587593 5 ==== osd_op_reply(27 [pgnls start_epoch 34 out=76b] v33'3 uv3 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x7fc4d8003040 con 0x7fc4d400c2e0 2024-09-17T04:34:16.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.090+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:28 4.1b 4:d8000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x7fc4d00622f0 2024-09-17T04:34:16.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.090+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 <== osd.4 v2:172.21.15.92:6818/2509785752 9 ==== osd_op_reply(28 [pgnls start_epoch 34 out=49b] v32'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc4d406a040 con 0x7fc4d00622f0 2024-09-17T04:34:16.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.090+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:29 4.7 4:e0000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x7fc4d4022da0 2024-09-17T04:34:16.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.091+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.1 v2:172.21.15.37:6824/2657955935 4 ==== osd_op_reply(29 [pgnls start_epoch 34 out=73b] v32'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7fc4d8003040 con 0x7fc4d4022da0 2024-09-17T04:34:16.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.091+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:30 4.17 4:e8000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x5630f088beb0 2024-09-17T04:34:16.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.091+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.3 v2:172.21.15.37:6800/809975826 7 ==== osd_op_reply(30 [pgnls start_epoch 34 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc4d8003040 con 0x5630f088beb0 2024-09-17T04:34:16.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.091+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:31 4.f 4:f0000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x7fc4d4022da0 2024-09-17T04:34:16.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.091+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.1 v2:172.21.15.37:6824/2657955935 5 ==== osd_op_reply(31 [pgnls start_epoch 34 out=73b] v32'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7fc4d8003040 con 0x7fc4d4022da0 2024-09-17T04:34:16.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.091+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] -- osd_op(unknown.0.0:32 4.1f 4:f8000000:%01:::head [pgnls start_epoch 34 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e34) -- 0x5630f0740490 con 0x7fc4d8021940 2024-09-17T04:34:16.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.092+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 <== osd.6 v2:172.21.15.92:6810/4078365985 3 ==== osd_op_reply(32 [pgnls start_epoch 34 out=101b] v33'6 uv6 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x7fc4d406a040 con 0x7fc4d8021940 2024-09-17T04:34:16.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.092+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:33 4.10 4:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e34) -- 0x5630f0758c00 con 0x5630f088beb0 2024-09-17T04:34:16.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.092+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:34 4.10 4:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e34) -- 0x5630f088b860 con 0x5630f088beb0 2024-09-17T04:34:16.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.092+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:35 4.10 4:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e34) -- 0x5630f088ecc0 con 0x5630f088beb0 2024-09-17T04:34:16.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.092+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:36 4.10 4:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e34) -- 0x5630f088f510 con 0x5630f088beb0 2024-09-17T04:34:16.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.092+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:37 4.10 4:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e34) -- 0x5630f088fd60 con 0x5630f088beb0 2024-09-17T04:34:16.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.092+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:38 4.14 4:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e34) -- 0x5630f08905b0 con 0x5630f088beb0 2024-09-17T04:34:16.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.092+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:39 4.14 4:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e34) -- 0x5630f0890e00 con 0x5630f088beb0 2024-09-17T04:34:16.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.093+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:40 4.14 4:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e34) -- 0x5630f0891650 con 0x5630f088beb0 2024-09-17T04:34:16.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.093+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:41 4.14 4:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e34) -- 0x5630f0891b00 con 0x5630f088beb0 2024-09-17T04:34:16.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.093+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:42 4.1c 4:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e34) -- 0x5630f0892200 con 0x7fc4d401f720 2024-09-17T04:34:16.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.093+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:43 4.6 4:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e34) -- 0x5630f0892e00 con 0x7fc4d400c2e0 2024-09-17T04:34:16.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.093+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:44 4.e 4:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e34) -- 0x5630f08936f0 con 0x7fc4d00622f0 2024-09-17T04:34:16.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.093+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:45 4.15 4:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e34) -- 0x5630f08940f0 con 0x7fc4d800e8d0 2024-09-17T04:34:16.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.094+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:46 4.15 4:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e34) -- 0x5630f0894c90 con 0x7fc4d800e8d0 2024-09-17T04:34:16.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.094+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:47 4.d 4:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e34) -- 0x5630f0895810 con 0x7fc4d00622f0 2024-09-17T04:34:16.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.094+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:48 4.1d 4:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e34) -- 0x5630f0896390 con 0x5630f088beb0 2024-09-17T04:34:16.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.095+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 <== osd.4 v2:172.21.15.92:6818/2509785752 10 ==== osd_op_reply(44 608.00000000 [delete] v34'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4d406a040 con 0x7fc4d00622f0 2024-09-17T04:34:16.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.095+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.3 v2:172.21.15.37:6800/809975826 8 ==== osd_op_reply(38 1.00000000.inode [delete] v34'20 uv20 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc4d8003040 con 0x5630f088beb0 2024-09-17T04:34:16.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.095+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.3 v2:172.21.15.37:6800/809975826 9 ==== osd_op_reply(33 601.00000000 [delete] v34'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4d8003040 con 0x5630f088beb0 2024-09-17T04:34:16.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.096+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:49 4.13 4:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e34) -- 0x5630f088af00 con 0x7fc4d00622f0 2024-09-17T04:34:16.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.096+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:50 4.13 4:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e34) -- 0x5630f08967b0 con 0x7fc4d00622f0 2024-09-17T04:34:16.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.095+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.3 v2:172.21.15.37:6800/809975826 10 ==== osd_op_reply(39 200.00000000 [delete] v34'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4d8003040 con 0x5630f088beb0 2024-09-17T04:34:16.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.096+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:51 4.b 4:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e34) -- 0x5630f0897000 con 0x7fc4d400c2e0 2024-09-17T04:34:16.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.096+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.3 v2:172.21.15.37:6800/809975826 11 ==== osd_op_reply(34 602.00000000 [delete] v34'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4d8003040 con 0x5630f088beb0 2024-09-17T04:34:16.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.096+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:52 4.7 4:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e34) -- 0x5630f0890950 con 0x7fc4d4022da0 2024-09-17T04:34:16.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.096+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:53 4.f 4:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e34) -- 0x5630f08974b0 con 0x7fc4d4022da0 2024-09-17T04:34:16.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.096+0000 7fc4df438640 1 -- 172.21.15.37:0/988470404 <== osd.5 v2:172.21.15.92:6826/248263832 3 ==== osd_op_reply(45 604.00000000 [delete] v34'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4d0009070 con 0x7fc4d800e8d0 2024-09-17T04:34:16.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.096+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] -- osd_op(unknown.0.0:54 4.1f 4:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e34) -- 0x5630f0890950 con 0x7fc4d8021940 2024-09-17T04:34:16.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.096+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 <== osd.4 v2:172.21.15.92:6818/2509785752 11 ==== osd_op_reply(47 mds_snaptable [delete] v34'2 uv2 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc4d406a040 con 0x7fc4d00622f0 2024-09-17T04:34:16.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.096+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 --> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] -- osd_op(unknown.0.0:55 4.1f 4:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e34) -- 0x5630f0890950 con 0x7fc4d8021940 2024-09-17T04:34:16.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.096+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.3 v2:172.21.15.37:6800/809975826 12 ==== osd_op_reply(35 10000000000.00000000 [delete] v34'13 uv13 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc4d8003040 con 0x5630f088beb0 2024-09-17T04:34:16.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.097+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.0 v2:172.21.15.37:6808/2283587593 6 ==== osd_op_reply(43 mds0_openfiles.0 [delete] v34'5 uv5 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc4d8003040 con 0x7fc4d400c2e0 2024-09-17T04:34:16.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.097+0000 7fc4df438640 1 -- 172.21.15.37:0/988470404 <== osd.5 v2:172.21.15.92:6826/248263832 4 ==== osd_op_reply(46 500.00000000 [delete] v34'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4d0009070 con 0x7fc4d800e8d0 2024-09-17T04:34:16.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.097+0000 7fc4df438640 1 -- 172.21.15.37:0/988470404 <== osd.2 v2:172.21.15.37:6816/579720176 2 ==== osd_op_reply(42 607.00000000 [delete] v34'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4d0009070 con 0x7fc4d401f720 2024-09-17T04:34:16.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.097+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.3 v2:172.21.15.37:6800/809975826 13 ==== osd_op_reply(40 200.00000001 [delete] v34'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4d8003040 con 0x5630f088beb0 2024-09-17T04:34:16.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.097+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.3 v2:172.21.15.37:6800/809975826 14 ==== osd_op_reply(36 600.00000000 [delete] v34'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4d8003040 con 0x5630f088beb0 2024-09-17T04:34:16.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.097+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.3 v2:172.21.15.37:6800/809975826 15 ==== osd_op_reply(48 605.00000000 [delete] v34'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4d8003040 con 0x5630f088beb0 2024-09-17T04:34:16.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.098+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.1 v2:172.21.15.37:6824/2657955935 6 ==== osd_op_reply(52 609.00000000 [delete] v34'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4d8003040 con 0x7fc4d4022da0 2024-09-17T04:34:16.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.098+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.1 v2:172.21.15.37:6824/2657955935 7 ==== osd_op_reply(53 400.00000000 [delete] v34'2 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4d8003040 con 0x7fc4d4022da0 2024-09-17T04:34:16.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.098+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 <== osd.4 v2:172.21.15.92:6818/2509785752 12 ==== osd_op_reply(49 mds0_inotable [delete] v34'4 uv4 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc4d406a040 con 0x7fc4d00622f0 2024-09-17T04:34:16.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.098+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.3 v2:172.21.15.37:6800/809975826 16 ==== osd_op_reply(37 603.00000000 [delete] v34'15 uv15 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4d8003040 con 0x5630f088beb0 2024-09-17T04:34:16.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.098+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.3 v2:172.21.15.37:6800/809975826 17 ==== osd_op_reply(41 606.00000000 [delete] v34'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4d8003040 con 0x5630f088beb0 2024-09-17T04:34:16.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.098+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 <== osd.6 v2:172.21.15.92:6810/4078365985 4 ==== osd_op_reply(54 100.00000000.inode [delete] v34'7 uv7 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc4d406a040 con 0x7fc4d8021940 2024-09-17T04:34:16.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.099+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 <== osd.6 v2:172.21.15.92:6810/4078365985 5 ==== osd_op_reply(55 1.00000000 [delete] v34'8 uv8 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc4d406a040 con 0x7fc4d8021940 2024-09-17T04:34:16.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.099+0000 7fc4dec37640 1 -- 172.21.15.37:0/988470404 <== osd.4 v2:172.21.15.92:6818/2509785752 13 ==== osd_op_reply(50 100.00000000 [delete] v34'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4d406a040 con 0x7fc4d00622f0 2024-09-17T04:34:16.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 <== osd.0 v2:172.21.15.37:6808/2283587593 7 ==== osd_op_reply(51 mds0_sessionmap [delete] v34'4 uv4 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fc4d8003040 con 0x7fc4d400c2e0 2024-09-17T04:34:16.102 INFO:teuthology.orchestra.run.smithi037.stdout:Removed 23 objects 2024-09-17T04:34:16.102 INFO:teuthology.orchestra.run.smithi037.stdout:successfully purged pool cephfs_metadata 2024-09-17T04:34:16.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x7fc4d400c2e0 msgr2=0x7fc4d407e920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:16.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4e660bf40 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x7fc4d400c2e0 0x7fc4d407e920 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:16.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x7fc4d4022da0 msgr2=0x7fc4d4056470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:16.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4e660bf40 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x7fc4d4022da0 0x7fc4d4056470 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:16.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x7fc4d401f720 msgr2=0x7fc4d4062410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:16.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4e660bf40 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x7fc4d401f720 0x7fc4d4062410 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:16.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x5630f088beb0 msgr2=0x5630f088e2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:16.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4e660bf40 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x5630f088beb0 0x5630f088e2d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:16.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x7fc4d00622f0 msgr2=0x7fc4d0060d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:16.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4e660bf40 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x7fc4d00622f0 0x7fc4d0060d60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:16.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x7fc4d800e8d0 msgr2=0x7fc4d800a2c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4e660bf40 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x7fc4d800e8d0 0x7fc4d800a2c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] conn(0x7fc4d8021940 msgr2=0x7fc4d80203c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4e660bf40 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] conn(0x7fc4d8021940 0x7fc4d80203c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 reap_dead start 2024-09-17T04:34:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 reap_dead start 2024-09-17T04:34:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 reap_dead start 2024-09-17T04:34:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc4cc077570 msgr2=0x7fc4cc079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4dfc39640 1 -- 172.21.15.37:0/988470404 reap_dead start 2024-09-17T04:34:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4e660bf40 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc4cc077570 0x7fc4cc079a80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4025de0 tx=0x7fc4d4064040 comp rx=0 tx=0).stop 2024-09-17T04:34:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5630f0710580 msgr2=0x5630f07ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4e660bf40 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5630f0710580 0x5630f07ce250 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0002e80 tx=0x7fc4d000c7a0 comp rx=0 tx=0).stop 2024-09-17T04:34:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 shutdown_connections 2024-09-17T04:34:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4e660bf40 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc4cc077570 0x7fc4cc079a80 secure :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4025de0 tx=0x7fc4d4064040 comp rx=0 tx=0).stop 2024-09-17T04:34:16.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4e660bf40 1 --2- 172.21.15.37:0/988470404 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5630f0710580 0x5630f07ce250 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0002e80 tx=0x7fc4d000c7a0 comp rx=0 tx=0).stop 2024-09-17T04:34:16.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 >> 172.21.15.37:0/988470404 conn(0x5630f0632ac0 msgr2=0x5630f07cfbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:16.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 shutdown_connections 2024-09-17T04:34:16.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.101+0000 7fc4e660bf40 1 -- 172.21.15.37:0/988470404 wait complete. 2024-09-17T04:34:16.106 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'recovery_fs' 2024-09-17T04:34:16.106 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create recovery_fs_metadata --pg_num_min 64 2024-09-17T04:34:16.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.232+0000 7fad1128b640 1 -- 172.21.15.37:0/2679415223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad0c0cd050 msgr2=0x7fad0c0d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:16.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.232+0000 7fad1128b640 1 --2- 172.21.15.37:0/2679415223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad0c0cd050 0x7fad0c0d00c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7facfc009860 tx=0x7facfc02a460 comp rx=0 tx=0).stop 2024-09-17T04:34:16.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.232+0000 7fad1128b640 1 -- 172.21.15.37:0/2679415223 shutdown_connections 2024-09-17T04:34:16.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.232+0000 7fad1128b640 1 --2- 172.21.15.37:0/2679415223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad0c0cd050 0x7fad0c0d00c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:16.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.232+0000 7fad1128b640 1 -- 172.21.15.37:0/2679415223 >> 172.21.15.37:0/2679415223 conn(0x7fad0c0cee70 msgr2=0x7fad0c0cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:16.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.232+0000 7fad1128b640 1 -- 172.21.15.37:0/2679415223 shutdown_connections 2024-09-17T04:34:16.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.232+0000 7fad1128b640 1 -- 172.21.15.37:0/2679415223 wait complete. 2024-09-17T04:34:16.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.234+0000 7fad1128b640 1 Processor -- start 2024-09-17T04:34:16.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.234+0000 7fad1128b640 1 -- start start 2024-09-17T04:34:16.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.235+0000 7fad1128b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad0c0b7820 0x7fad0c1f6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:16.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.235+0000 7fad1128b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fad0c0b7c30 0x7fad0c1f98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:16.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.235+0000 7fad1128b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad0c0cd050 0x7fad0c1f9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:16.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.235+0000 7fad1128b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c0d1120 con 0x7fad0c0b7820 2024-09-17T04:34:16.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.235+0000 7fad1128b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c0d12a0 con 0x7fad0c0cd050 2024-09-17T04:34:16.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.235+0000 7fad1128b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fad0c0d1420 con 0x7fad0c0b7c30 2024-09-17T04:34:16.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.235+0000 7fad0affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad0c0b7820 0x7fad0c1f6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:16.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.235+0000 7fad0affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad0c0b7820 0x7fad0c1f6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38410/0 (socket says 172.21.15.37:38410) 2024-09-17T04:34:16.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.235+0000 7fad0affd640 1 -- 172.21.15.37:0/3296627869 learned_addr learned my addr 172.21.15.37:0/3296627869 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:16.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.235+0000 7fad0a7fc640 1 --2- 172.21.15.37:0/3296627869 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fad0c0b7c30 0x7fad0c1f98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:16.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.236+0000 7fad0b7fe640 1 --2- 172.21.15.37:0/3296627869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad0c0cd050 0x7fad0c1f9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:16.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.236+0000 7fad0affd640 1 -- 172.21.15.37:0/3296627869 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fad0c0b7c30 msgr2=0x7fad0c1f98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:16.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.236+0000 7fad0affd640 1 --2- 172.21.15.37:0/3296627869 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fad0c0b7c30 0x7fad0c1f98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:16.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.236+0000 7fad0affd640 1 -- 172.21.15.37:0/3296627869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad0c0cd050 msgr2=0x7fad0c1f9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:16.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.236+0000 7fad0affd640 1 --2- 172.21.15.37:0/3296627869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad0c0cd050 0x7fad0c1f9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:16.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.236+0000 7fad0affd640 1 -- 172.21.15.37:0/3296627869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad0c06ea10 con 0x7fad0c0b7820 2024-09-17T04:34:16.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.236+0000 7fad0a7fc640 1 --2- 172.21.15.37:0/3296627869 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fad0c0b7c30 0x7fad0c1f98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:16.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.236+0000 7fad0b7fe640 1 --2- 172.21.15.37:0/3296627869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad0c0cd050 0x7fad0c1f9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:16.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.236+0000 7fad0affd640 1 --2- 172.21.15.37:0/3296627869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad0c0b7820 0x7fad0c1f6bf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7facfc009780 tx=0x7facfc003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:16.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.236+0000 7fad037fe640 1 -- 172.21.15.37:0/3296627869 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc037040 con 0x7fad0c0b7820 2024-09-17T04:34:16.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.236+0000 7fad037fe640 1 -- 172.21.15.37:0/3296627869 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facfc040070 con 0x7fad0c0b7820 2024-09-17T04:34:16.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.236+0000 7fad1128b640 1 -- 172.21.15.37:0/3296627869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad0c06ecb0 con 0x7fad0c0b7820 2024-09-17T04:34:16.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.236+0000 7fad037fe640 1 -- 172.21.15.37:0/3296627869 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc005c90 con 0x7fad0c0b7820 2024-09-17T04:34:16.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.236+0000 7fad1128b640 1 -- 172.21.15.37:0/3296627869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad0c06f270 con 0x7fad0c0b7820 2024-09-17T04:34:16.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.237+0000 7fad1128b640 1 -- 172.21.15.37:0/3296627869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad0c0d00c0 con 0x7fad0c0b7820 2024-09-17T04:34:16.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.238+0000 7fad037fe640 1 -- 172.21.15.37:0/3296627869 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7facfc0326a0 con 0x7fad0c0b7820 2024-09-17T04:34:16.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.238+0000 7fad037fe640 1 --2- 172.21.15.37:0/3296627869 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7facec078bc0 0x7facec07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:16.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.239+0000 7fad037fe640 1 -- 172.21.15.37:0/3296627869 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5795+0+0 (secure 0 0 0) 0x7facfc0baf60 con 0x7fad0c0b7820 2024-09-17T04:34:16.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.239+0000 7fad0a7fc640 1 --2- 172.21.15.37:0/3296627869 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7facec078bc0 0x7facec07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:16.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.239+0000 7fad0a7fc640 1 --2- 172.21.15.37:0/3296627869 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7facec078bc0 0x7facec07b0d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7facf4001480 tx=0x7facf4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:16.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.243+0000 7fad037fe640 1 -- 172.21.15.37:0/3296627869 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7facfc083340 con 0x7fad0c0b7820 2024-09-17T04:34:16.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:16.431+0000 7fad1128b640 1 -- 172.21.15.37:0/3296627869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "recovery_fs_metadata", "pg_num_min": 64} v 0) -- 0x7fad0c0af6c0 con 0x7fad0c0b7820 2024-09-17T04:34:17.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.203+0000 7fad037fe640 1 -- 172.21.15.37:0/3296627869 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "recovery_fs_metadata", "pg_num_min": 64}]=0 pool 'recovery_fs_metadata' created v39) ==== 148+0+0 (secure 0 0 0) 0x7facfc082c40 con 0x7fad0c0b7820 2024-09-17T04:34:17.204 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'recovery_fs_metadata' created 2024-09-17T04:34:17.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.207+0000 7fad1128b640 1 -- 172.21.15.37:0/3296627869 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7facec078bc0 msgr2=0x7facec07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:17.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.207+0000 7fad1128b640 1 --2- 172.21.15.37:0/3296627869 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7facec078bc0 0x7facec07b0d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7facf4001480 tx=0x7facf4009040 comp rx=0 tx=0).stop 2024-09-17T04:34:17.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.207+0000 7fad1128b640 1 -- 172.21.15.37:0/3296627869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad0c0b7820 msgr2=0x7fad0c1f6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:17.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.207+0000 7fad1128b640 1 --2- 172.21.15.37:0/3296627869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad0c0b7820 0x7fad0c1f6bf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7facfc009780 tx=0x7facfc003b60 comp rx=0 tx=0).stop 2024-09-17T04:34:17.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.207+0000 7fad1128b640 1 -- 172.21.15.37:0/3296627869 shutdown_connections 2024-09-17T04:34:17.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.207+0000 7fad1128b640 1 --2- 172.21.15.37:0/3296627869 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7facec078bc0 0x7facec07b0d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:17.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.207+0000 7fad1128b640 1 --2- 172.21.15.37:0/3296627869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad0c0cd050 0x7fad0c1f9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:17.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.207+0000 7fad1128b640 1 --2- 172.21.15.37:0/3296627869 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fad0c0b7c30 0x7fad0c1f98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:17.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.207+0000 7fad1128b640 1 --2- 172.21.15.37:0/3296627869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad0c0b7820 0x7fad0c1f6bf0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:17.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.207+0000 7fad1128b640 1 -- 172.21.15.37:0/3296627869 >> 172.21.15.37:0/3296627869 conn(0x7fad0c0cee70 msgr2=0x7fad0c071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:17.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.208+0000 7fad1128b640 1 -- 172.21.15.37:0/3296627869 shutdown_connections 2024-09-17T04:34:17.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.208+0000 7fad1128b640 1 -- 172.21.15.37:0/3296627869 wait complete. 2024-09-17T04:34:17.227 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T04:34:17.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.359+0000 7fc9a49f1640 1 -- 172.21.15.37:0/461648698 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc9a00d1e70 msgr2=0x7fc9a00cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:17.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.359+0000 7fc9a49f1640 1 --2- 172.21.15.37:0/461648698 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc9a00d1e70 0x7fc9a00cdc70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc998009830 tx=0x7fc99802a510 comp rx=0 tx=0).stop 2024-09-17T04:34:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.359+0000 7fc9a49f1640 1 -- 172.21.15.37:0/461648698 shutdown_connections 2024-09-17T04:34:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.359+0000 7fc9a49f1640 1 --2- 172.21.15.37:0/461648698 >> v2:172.21.15.92:3300/0 conn(0x7fc9a00b5920 0x7fc9a0160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.359+0000 7fc9a49f1640 1 --2- 172.21.15.37:0/461648698 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc9a00d1e70 0x7fc9a00cdc70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.359+0000 7fc9a49f1640 1 -- 172.21.15.37:0/461648698 >> 172.21.15.37:0/461648698 conn(0x7fc9a00d1350 msgr2=0x7fc9a00d1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.359+0000 7fc9a49f1640 1 -- 172.21.15.37:0/461648698 shutdown_connections 2024-09-17T04:34:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.359+0000 7fc9a49f1640 1 -- 172.21.15.37:0/461648698 wait complete. 2024-09-17T04:34:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.361+0000 7fc9a49f1640 1 Processor -- start 2024-09-17T04:34:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.361+0000 7fc9a49f1640 1 -- start start 2024-09-17T04:34:17.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.362+0000 7fc9a49f1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9a00b5920 0x7fc9a014ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:17.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.362+0000 7fc9a49f1640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc9a00ce1b0 0x7fc9a014b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:17.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.362+0000 7fc9a49f1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a00d1e70 0x7fc9a01508a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:17.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.362+0000 7fc9a49f1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a006d1d0 con 0x7fc9a00d1e70 2024-09-17T04:34:17.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.362+0000 7fc9a49f1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a006d050 con 0x7fc9a00b5920 2024-09-17T04:34:17.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.362+0000 7fc9a49f1640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fc9a006d350 con 0x7fc9a00ce1b0 2024-09-17T04:34:17.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.363+0000 7fc99ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a00d1e70 0x7fc9a01508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:17.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.363+0000 7fc99ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a00d1e70 0x7fc9a01508a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38424/0 (socket says 172.21.15.37:38424) 2024-09-17T04:34:17.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.363+0000 7fc99ed9d640 1 -- 172.21.15.37:0/2052173112 learned_addr learned my addr 172.21.15.37:0/2052173112 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:17.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.363+0000 7fc99e59c640 1 --2- 172.21.15.37:0/2052173112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9a00b5920 0x7fc9a014ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:17.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.363+0000 7fc99dd9b640 1 --2- 172.21.15.37:0/2052173112 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc9a00ce1b0 0x7fc9a014b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:17.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.363+0000 7fc99ed9d640 1 -- 172.21.15.37:0/2052173112 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc9a00ce1b0 msgr2=0x7fc9a014b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:17.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.363+0000 7fc99ed9d640 1 --2- 172.21.15.37:0/2052173112 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc9a00ce1b0 0x7fc9a014b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:17.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.363+0000 7fc99ed9d640 1 -- 172.21.15.37:0/2052173112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9a00b5920 msgr2=0x7fc9a014ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:17.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.363+0000 7fc99ed9d640 1 --2- 172.21.15.37:0/2052173112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9a00b5920 0x7fc9a014ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:17.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.363+0000 7fc99ed9d640 1 -- 172.21.15.37:0/2052173112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a015c4a0 con 0x7fc9a00d1e70 2024-09-17T04:34:17.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.363+0000 7fc99ed9d640 1 --2- 172.21.15.37:0/2052173112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a00d1e70 0x7fc9a01508a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc99000ec20 tx=0x7fc99000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:17.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.363+0000 7fc996ffd640 1 -- 172.21.15.37:0/2052173112 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc990003b70 con 0x7fc9a00d1e70 2024-09-17T04:34:17.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.363+0000 7fc996ffd640 1 -- 172.21.15.37:0/2052173112 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc990003e00 con 0x7fc9a00d1e70 2024-09-17T04:34:17.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.363+0000 7fc9a49f1640 1 -- 172.21.15.37:0/2052173112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9a014b080 con 0x7fc9a00d1e70 2024-09-17T04:34:17.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.363+0000 7fc996ffd640 1 -- 172.21.15.37:0/2052173112 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc990004280 con 0x7fc9a00d1e70 2024-09-17T04:34:17.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.363+0000 7fc9a49f1640 1 -- 172.21.15.37:0/2052173112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9a014fae0 con 0x7fc9a00d1e70 2024-09-17T04:34:17.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.365+0000 7fc9a49f1640 1 -- 172.21.15.37:0/2052173112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9a00cdc70 con 0x7fc9a00d1e70 2024-09-17T04:34:17.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.365+0000 7fc996ffd640 1 -- 172.21.15.37:0/2052173112 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc990004420 con 0x7fc9a00d1e70 2024-09-17T04:34:17.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.366+0000 7fc996ffd640 1 --2- 172.21.15.37:0/2052173112 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc980078b90 0x7fc98007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:17.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.366+0000 7fc996ffd640 1 -- 172.21.15.37:0/2052173112 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6170+0+0 (secure 0 0 0) 0x7fc99009bd30 con 0x7fc9a00d1e70 2024-09-17T04:34:17.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.366+0000 7fc99e59c640 1 --2- 172.21.15.37:0/2052173112 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc980078b90 0x7fc98007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:17.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.367+0000 7fc99e59c640 1 --2- 172.21.15.37:0/2052173112 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc980078b90 0x7fc98007b0a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc998034000 tx=0x7fc998004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:17.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.371+0000 7fc996ffd640 1 -- 172.21.15.37:0/2052173112 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc9a014b080 con 0x7fc9a00d1e70 2024-09-17T04:34:17.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.555+0000 7fc9a49f1640 1 -- 172.21.15.37:0/2052173112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc9a0154090 con 0x7fc9a00d1e70 2024-09-17T04:34:17.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.556+0000 7fc996ffd640 1 -- 172.21.15.37:0/2052173112 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' already exists v39) ==== 177+0+0 (secure 0 0 0) 0x7fc9a015c4a0 con 0x7fc9a00d1e70 2024-09-17T04:34:17.557 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' already exists 2024-09-17T04:34:17.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.558+0000 7fc9a49f1640 1 -- 172.21.15.37:0/2052173112 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc980078b90 msgr2=0x7fc98007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:17.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.558+0000 7fc9a49f1640 1 --2- 172.21.15.37:0/2052173112 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc980078b90 0x7fc98007b0a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc998034000 tx=0x7fc998004730 comp rx=0 tx=0).stop 2024-09-17T04:34:17.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.558+0000 7fc9a49f1640 1 -- 172.21.15.37:0/2052173112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a00d1e70 msgr2=0x7fc9a01508a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:17.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.558+0000 7fc9a49f1640 1 --2- 172.21.15.37:0/2052173112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a00d1e70 0x7fc9a01508a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc99000ec20 tx=0x7fc99000c6a0 comp rx=0 tx=0).stop 2024-09-17T04:34:17.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.558+0000 7fc9a49f1640 1 -- 172.21.15.37:0/2052173112 shutdown_connections 2024-09-17T04:34:17.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.558+0000 7fc9a49f1640 1 --2- 172.21.15.37:0/2052173112 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc980078b90 0x7fc98007b0a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:17.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.558+0000 7fc9a49f1640 1 --2- 172.21.15.37:0/2052173112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a00d1e70 0x7fc9a01508a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:17.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.558+0000 7fc9a49f1640 1 --2- 172.21.15.37:0/2052173112 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc9a00ce1b0 0x7fc9a014b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:17.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.558+0000 7fc9a49f1640 1 --2- 172.21.15.37:0/2052173112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9a00b5920 0x7fc9a014ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:17.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.558+0000 7fc9a49f1640 1 -- 172.21.15.37:0/2052173112 >> 172.21.15.37:0/2052173112 conn(0x7fc9a00d1350 msgr2=0x7fc9a00caf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:17.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.558+0000 7fc9a49f1640 1 -- 172.21.15.37:0/2052173112 shutdown_connections 2024-09-17T04:34:17.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.558+0000 7fc9a49f1640 1 -- 172.21.15.37:0/2052173112 wait complete. 2024-09-17T04:34:17.570 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new recovery_fs recovery_fs_metadata cephfs_data --recover --allow-dangerous-metadata-overlay 2024-09-17T04:34:17.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.692+0000 7f95c679d640 1 -- 172.21.15.37:0/1225338202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95c00b3cd0 msgr2=0x7f95c00b40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:17.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.692+0000 7f95c679d640 1 --2- 172.21.15.37:0/1225338202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95c00b3cd0 0x7f95c00b40e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f95b8005fd0 tx=0x7f95b802a710 comp rx=0 tx=0).stop 2024-09-17T04:34:17.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.692+0000 7f95c679d640 1 -- 172.21.15.37:0/1225338202 shutdown_connections 2024-09-17T04:34:17.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.692+0000 7f95c679d640 1 --2- 172.21.15.37:0/1225338202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95c00b3cd0 0x7f95c00b40e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:17.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.692+0000 7f95c679d640 1 --2- 172.21.15.37:0/1225338202 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f95c00b33d0 0x7f95c00b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:17.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.692+0000 7f95c679d640 1 -- 172.21.15.37:0/1225338202 >> 172.21.15.37:0/1225338202 conn(0x7f95c009d700 msgr2=0x7f95c00d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:17.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.692+0000 7f95c679d640 1 -- 172.21.15.37:0/1225338202 shutdown_connections 2024-09-17T04:34:17.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.692+0000 7f95c679d640 1 -- 172.21.15.37:0/1225338202 wait complete. 2024-09-17T04:34:17.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.695+0000 7f95c679d640 1 Processor -- start 2024-09-17T04:34:17.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.695+0000 7f95c679d640 1 -- start start 2024-09-17T04:34:17.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.695+0000 7f95c679d640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f95c00b3cd0 0x7f95c01de430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:17.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.695+0000 7f95c679d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c00cf010 0x7f95c01f5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:17.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.695+0000 7f95c679d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95c01e3a30 0x7f95c01e0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:17.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.695+0000 7f95c679d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f95c00b47a0 con 0x7f95c00cf010 2024-09-17T04:34:17.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.695+0000 7f95c679d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f95c00b4620 con 0x7f95c01e3a30 2024-09-17T04:34:17.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.695+0000 7f95c679d640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f95c00b4920 con 0x7f95c00b3cd0 2024-09-17T04:34:17.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.695+0000 7f95c4f9a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c00cf010 0x7f95c01f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:17.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.696+0000 7f95c4f9a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c00cf010 0x7f95c01f5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38438/0 (socket says 172.21.15.37:38438) 2024-09-17T04:34:17.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.696+0000 7f95c4f9a640 1 -- 172.21.15.37:0/1354751769 learned_addr learned my addr 172.21.15.37:0/1354751769 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:17.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.696+0000 7f95c5f9c640 1 --2- 172.21.15.37:0/1354751769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95c01e3a30 0x7f95c01e0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:17.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.696+0000 7f95c579b640 1 --2- 172.21.15.37:0/1354751769 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f95c00b3cd0 0x7f95c01de430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:17.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.696+0000 7f95c4f9a640 1 -- 172.21.15.37:0/1354751769 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f95c00b3cd0 msgr2=0x7f95c01de430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:17.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.696+0000 7f95c4f9a640 1 --2- 172.21.15.37:0/1354751769 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f95c00b3cd0 0x7f95c01de430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:17.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.696+0000 7f95c4f9a640 1 -- 172.21.15.37:0/1354751769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95c01e3a30 msgr2=0x7f95c01e0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:17.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.696+0000 7f95c4f9a640 1 --2- 172.21.15.37:0/1354751769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95c01e3a30 0x7f95c01e0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:17.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.696+0000 7f95c4f9a640 1 -- 172.21.15.37:0/1354751769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c01f5e50 con 0x7f95c00cf010 2024-09-17T04:34:17.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.696+0000 7f95c4f9a640 1 --2- 172.21.15.37:0/1354751769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c00cf010 0x7f95c01f5320 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f95b000c500 tx=0x7f95b000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:17.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.696+0000 7f95b5ffb640 1 -- 172.21.15.37:0/1354751769 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b0016020 con 0x7f95c00cf010 2024-09-17T04:34:17.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.696+0000 7f95b5ffb640 1 -- 172.21.15.37:0/1354751769 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95b0012920 con 0x7f95c00cf010 2024-09-17T04:34:17.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.696+0000 7f95c679d640 1 -- 172.21.15.37:0/1354751769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95c01f6130 con 0x7f95c00cf010 2024-09-17T04:34:17.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.696+0000 7f95b5ffb640 1 -- 172.21.15.37:0/1354751769 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b0012d70 con 0x7f95c00cf010 2024-09-17T04:34:17.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.697+0000 7f95c679d640 1 -- 172.21.15.37:0/1354751769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95c01dec60 con 0x7f95c00cf010 2024-09-17T04:34:17.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.697+0000 7f95c679d640 1 -- 172.21.15.37:0/1354751769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95c01f5bc0 con 0x7f95c00cf010 2024-09-17T04:34:17.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.698+0000 7f95b5ffb640 1 -- 172.21.15.37:0/1354751769 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f95b0019020 con 0x7f95c00cf010 2024-09-17T04:34:17.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.698+0000 7f95b5ffb640 1 --2- 172.21.15.37:0/1354751769 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f95a4078b90 0x7f95a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:17.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.699+0000 7f95b5ffb640 1 -- 172.21.15.37:0/1354751769 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6170+0+0 (secure 0 0 0) 0x7f95b009aec0 con 0x7f95c00cf010 2024-09-17T04:34:17.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.699+0000 7f95c579b640 1 --2- 172.21.15.37:0/1354751769 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f95a4078b90 0x7f95a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:17.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.699+0000 7f95c579b640 1 --2- 172.21.15.37:0/1354751769 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f95a4078b90 0x7f95a407b0a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f95bc009c00 tx=0x7f95bc005f30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:17.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.704+0000 7f95b5ffb640 1 -- 172.21.15.37:0/1354751769 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f95b0020020 con 0x7f95c00cf010 2024-09-17T04:34:17.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:17.926+0000 7f95c679d640 1 -- 172.21.15.37:0/1354751769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "recovery_fs", "metadata": "recovery_fs_metadata", "data": "cephfs_data", "recover": true, "allow_dangerous_metadata_overlay": true} v 0) -- 0x7f95c01e3020 con 0x7f95c00cf010 2024-09-17T04:34:18.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.204+0000 7f95b5ffb640 1 -- 172.21.15.37:0/1354751769 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "recovery_fs", "metadata": "recovery_fs_metadata", "data": "cephfs_data", "recover": true, "allow_dangerous_metadata_overlay": true}]=0 new fs with metadata pool 6 and data pool 5 v20) ==== 241+0+0 (secure 0 0 0) 0x7f95b0062af0 con 0x7f95c00cf010 2024-09-17T04:34:18.205 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 6 and data pool 5 2024-09-17T04:34:18.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.207+0000 7f95c679d640 1 -- 172.21.15.37:0/1354751769 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f95a4078b90 msgr2=0x7f95a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:18.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.207+0000 7f95c679d640 1 --2- 172.21.15.37:0/1354751769 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f95a4078b90 0x7f95a407b0a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f95bc009c00 tx=0x7f95bc005f30 comp rx=0 tx=0).stop 2024-09-17T04:34:18.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.207+0000 7f95c679d640 1 -- 172.21.15.37:0/1354751769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c00cf010 msgr2=0x7f95c01f5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:18.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.207+0000 7f95c679d640 1 --2- 172.21.15.37:0/1354751769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c00cf010 0x7f95c01f5320 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f95b000c500 tx=0x7f95b000c9c0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.207+0000 7f95c679d640 1 -- 172.21.15.37:0/1354751769 shutdown_connections 2024-09-17T04:34:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.207+0000 7f95c679d640 1 --2- 172.21.15.37:0/1354751769 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f95a4078b90 0x7f95a407b0a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.207+0000 7f95c679d640 1 --2- 172.21.15.37:0/1354751769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95c01e3a30 0x7f95c01e0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.207+0000 7f95c679d640 1 --2- 172.21.15.37:0/1354751769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c00cf010 0x7f95c01f5320 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.207+0000 7f95c679d640 1 --2- 172.21.15.37:0/1354751769 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f95c00b3cd0 0x7f95c01de430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.207+0000 7f95c679d640 1 -- 172.21.15.37:0/1354751769 >> 172.21.15.37:0/1354751769 conn(0x7f95c009d700 msgr2=0x7f95c00b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.207+0000 7f95c679d640 1 -- 172.21.15.37:0/1354751769 shutdown_connections 2024-09-17T04:34:18.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.207+0000 7f95c679d640 1 -- 172.21.15.37:0/1354751769 wait complete. 2024-09-17T04:34:18.220 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:34:18.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.336+0000 7f55d2d19640 1 -- 172.21.15.37:0/3959335495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55cc0cafa0 msgr2=0x7f55cc0b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:18.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.336+0000 7f55d2d19640 1 --2- 172.21.15.37:0/3959335495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55cc0cafa0 0x7f55cc0b3090 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f55cc072d70 tx=0x7f55bc02a560 comp rx=0 tx=0).stop 2024-09-17T04:34:18.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.336+0000 7f55d2d19640 1 -- 172.21.15.37:0/3959335495 shutdown_connections 2024-09-17T04:34:18.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.336+0000 7f55d2d19640 1 --2- 172.21.15.37:0/3959335495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55cc0cafa0 0x7f55cc0b3090 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.336+0000 7f55d2d19640 1 --2- 172.21.15.37:0/3959335495 >> v2:172.21.15.92:3300/0 conn(0x7f55cc070010 0x7f55cc0b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.336+0000 7f55d2d19640 1 -- 172.21.15.37:0/3959335495 >> 172.21.15.37:0/3959335495 conn(0x7f55cc06d3f0 msgr2=0x7f55cc0d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:18.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.336+0000 7f55d2d19640 1 -- 172.21.15.37:0/3959335495 shutdown_connections 2024-09-17T04:34:18.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.336+0000 7f55d2d19640 1 -- 172.21.15.37:0/3959335495 wait complete. 2024-09-17T04:34:18.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.338+0000 7f55d2d19640 1 Processor -- start 2024-09-17T04:34:18.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.338+0000 7f55d2d19640 1 -- start start 2024-09-17T04:34:18.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.339+0000 7f55d2d19640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55cc070010 0x7f55cc09cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:18.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.339+0000 7f55d2d19640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55cc0af7a0 0x7f55cc151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:18.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.339+0000 7f55d2d19640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55cc0cafa0 0x7f55cc15d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:18.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.339+0000 7f55d2d19640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc0b3b20 con 0x7f55cc070010 2024-09-17T04:34:18.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.339+0000 7f55d2d19640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc0b39a0 con 0x7f55cc0af7a0 2024-09-17T04:34:18.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.339+0000 7f55d2d19640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f55cc0b3ca0 con 0x7f55cc0cafa0 2024-09-17T04:34:18.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.339+0000 7f55d0ab5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55cc070010 0x7f55cc09cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:18.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.339+0000 7f55d0ab5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55cc070010 0x7f55cc09cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38464/0 (socket says 172.21.15.37:38464) 2024-09-17T04:34:18.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.339+0000 7f55d0ab5640 1 -- 172.21.15.37:0/2766004509 learned_addr learned my addr 172.21.15.37:0/2766004509 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:18.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.339+0000 7f55cbfff640 1 --2- 172.21.15.37:0/2766004509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55cc0af7a0 0x7f55cc151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:18.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.340+0000 7f55d12b6640 1 --2- 172.21.15.37:0/2766004509 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55cc0cafa0 0x7f55cc15d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:18.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.340+0000 7f55d0ab5640 1 -- 172.21.15.37:0/2766004509 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55cc0cafa0 msgr2=0x7f55cc15d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:18.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.340+0000 7f55d0ab5640 1 --2- 172.21.15.37:0/2766004509 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55cc0cafa0 0x7f55cc15d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.340+0000 7f55d0ab5640 1 -- 172.21.15.37:0/2766004509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55cc0af7a0 msgr2=0x7f55cc151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:18.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.340+0000 7f55d0ab5640 1 --2- 172.21.15.37:0/2766004509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55cc0af7a0 0x7f55cc151170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.340+0000 7f55d0ab5640 1 -- 172.21.15.37:0/2766004509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55cc09a0d0 con 0x7f55cc070010 2024-09-17T04:34:18.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.340+0000 7f55d0ab5640 1 --2- 172.21.15.37:0/2766004509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55cc070010 0x7f55cc09cec0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f55c400ba50 tx=0x7f55c400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:18.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.340+0000 7f55c97fa640 1 -- 172.21.15.37:0/2766004509 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c4019070 con 0x7f55cc070010 2024-09-17T04:34:18.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.340+0000 7f55c97fa640 1 -- 172.21.15.37:0/2766004509 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55c400cd10 con 0x7f55cc070010 2024-09-17T04:34:18.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.340+0000 7f55d2d19640 1 -- 172.21.15.37:0/2766004509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55cc1516b0 con 0x7f55cc070010 2024-09-17T04:34:18.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.340+0000 7f55c97fa640 1 -- 172.21.15.37:0/2766004509 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c40146a0 con 0x7f55cc070010 2024-09-17T04:34:18.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.340+0000 7f55d2d19640 1 -- 172.21.15.37:0/2766004509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55cc09a370 con 0x7f55cc070010 2024-09-17T04:34:18.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.341+0000 7f55d2d19640 1 -- 172.21.15.37:0/2766004509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55cc0b4170 con 0x7f55cc070010 2024-09-17T04:34:18.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.342+0000 7f55c97fa640 1 -- 172.21.15.37:0/2766004509 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f55c401c070 con 0x7f55cc070010 2024-09-17T04:34:18.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.342+0000 7f55c97fa640 1 --2- 172.21.15.37:0/2766004509 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f55b8078bc0 0x7f55b807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:18.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.342+0000 7f55c97fa640 1 -- 172.21.15.37:0/2766004509 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6248+0+0 (secure 0 0 0) 0x7f55c409acc0 con 0x7f55cc070010 2024-09-17T04:34:18.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.343+0000 7f55cbfff640 1 --2- 172.21.15.37:0/2766004509 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f55b8078bc0 0x7f55b807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:18.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.343+0000 7f55cbfff640 1 --2- 172.21.15.37:0/2766004509 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f55b8078bc0 0x7f55b807b0d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0d0fe0 tx=0x7f55c000a250 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:18.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.347+0000 7f55c97fa640 1 -- 172.21.15.37:0/2766004509 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f55c40a1080 con 0x7f55cc070010 2024-09-17T04:34:18.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.534+0000 7f55d2d19640 1 -- 172.21.15.37:0/2766004509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55cc0afd40 con 0x7f55cc070010 2024-09-17T04:34:18.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.535+0000 7f55c97fa640 1 -- 172.21.15.37:0/2766004509 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+18717 (secure 0 0 0) 0x7f55c4062800 con 0x7f55cc070010 2024-09-17T04:34:18.535 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:18.535 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":40,"fsid":"a908a603-430c-4209-8294-0c3460b35f42","created":"2024-09-17T04:32:54.139221+0000","modified":"2024-09-17T04:34:17.928819+0000","last_up_change":"2024-09-17T04:33:06.922210+0000","last_in_change":"2024-09-17T04:32:57.754118+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:33:08.461907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:33:57.558905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:58.446650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"recovery_fs_metadata","create_time":"2024-09-17T04:34:16.436885+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ea4b8d1-0d22-4dd9-b913-9df585ef24ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6809","nonce":2283587593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6811","nonce":2283587593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6815","nonce":2283587593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6813","nonce":2283587593}]},"public_addr":"172.21.15.37:6809/2283587593","cluster_addr":"172.21.15.37:6811/2283587593","heartbeat_back_addr":"172.21.15.37:6815/2283587593","heartbeat_front_addr":"172.21.15.37:6813/2283587593","state":["exists","up"]},{"osd":1,"uuid":"ac745a33-e8ad-4083-9c18-71bf81982b76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6825","nonce":2657955935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6827","nonce":2657955935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6831","nonce":2657955935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6829","nonce":2657955935}]},"public_addr":"172.21.15.37:6825/2657955935","cluster_addr":"172.21.15.37:6827/2657955935","heartbeat_back_addr":"172.21.15.37:6831/2657955935","heartbeat_front_addr":"172.21.15.37:6829/2657955935","state":["exists","up"]},{"osd":2,"uuid":"3c63f3d8-132a-4735-b0bc-d64313a46503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6817","nonce":579720176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6819","nonce":579720176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6823","nonce":579720176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6821","nonce":579720176}]},"public_addr":"172.21.15.37:6817/579720176","cluster_addr":"172.21.15.37:6819/579720176","heartbeat_back_addr":"172.21.15.37:6823/579720176","heartbeat_front_addr":"172.21.15.37:6821/579720176","state":["exists","up"]},{"osd":3,"uuid":"4f8c6ca2-ea1d-4610-9127-52c4dd8bc8e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6801","nonce":809975826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6803","nonce":809975826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6807","nonce":809975826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6805","nonce":809975826}]},"public_addr":"172.21.15.37:6801/809975826","cluster_addr":"172.21.15.37:6803/809975826","heartbeat_back_addr":"172.21.15.37:6807/809975826","heartbeat_front_addr":"172.21.15.37:6805/809975826","state":["exists","up"]},{"osd":4,"uuid":"63d0c4ab-f115-47b4-9611-12a10153473a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6819","nonce":2509785752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6821","nonce":2509785752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6825","nonce":2509785752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6823","nonce":2509785752}]},"public_addr":"172.21.15.92:6819/2509785752","cluster_addr":"172.21.15.92:6821/2509785752","heartbeat_back_addr":"172.21.15.92:6825/2509785752","heartbeat_front_addr":"172.21.15.92:6823/2509785752","state":["exists","up"]},{"osd":5,"uuid":"4c56c04d-0c07-4f76-9d75-6335121310ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6827","nonce":248263832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6829","nonce":248263832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6833","nonce":248263832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6831","nonce":248263832}]},"public_addr":"172.21.15.92:6827/248263832","cluster_addr":"172.21.15.92:6829/248263832","heartbeat_back_addr":"172.21.15.92:6833/248263832","heartbeat_front_addr":"172.21.15.92:6831/248263832","state":["exists","up"]},{"osd":6,"uuid":"b30eae7c-096a-4d24-ac51-96b8b13ecaee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6811","nonce":4078365985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6813","nonce":4078365985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6817","nonce":4078365985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6815","nonce":4078365985}]},"public_addr":"172.21.15.92:6811/4078365985","cluster_addr":"172.21.15.92:6813/4078365985","heartbeat_back_addr":"172.21.15.92:6817/4078365985","heartbeat_front_addr":"172.21.15.92:6815/4078365985","state":["exists","up"]},{"osd":7,"uuid":"62f12540-d374-4a4c-9ff7-63b7304fe77f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6803","nonce":3444787223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6805","nonce":3444787223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6809","nonce":3444787223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6807","nonce":3444787223}]},"public_addr":"172.21.15.92:6803/3444787223","cluster_addr":"172.21.15.92:6805/3444787223","heartbeat_back_addr":"172.21.15.92:6809/3444787223","heartbeat_front_addr":"172.21.15.92:6807/3444787223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.006420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.111135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.103325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.072656+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.200989+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.060455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.774773+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.865778+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/1358600605":"2024-09-18T04:34:15.699884+0000","172.21.15.37:6833/1358600605":"2024-09-18T04:34:15.699884+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:34:18.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.537+0000 7f55d2d19640 1 -- 172.21.15.37:0/2766004509 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f55b8078bc0 msgr2=0x7f55b807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:18.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.537+0000 7f55d2d19640 1 --2- 172.21.15.37:0/2766004509 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f55b8078bc0 0x7f55b807b0d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0d0fe0 tx=0x7f55c000a250 comp rx=0 tx=0).stop 2024-09-17T04:34:18.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.537+0000 7f55d2d19640 1 -- 172.21.15.37:0/2766004509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55cc070010 msgr2=0x7f55cc09cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:18.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.537+0000 7f55d2d19640 1 --2- 172.21.15.37:0/2766004509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55cc070010 0x7f55cc09cec0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f55c400ba50 tx=0x7f55c400bf10 comp rx=0 tx=0).stop 2024-09-17T04:34:18.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.537+0000 7f55d2d19640 1 -- 172.21.15.37:0/2766004509 shutdown_connections 2024-09-17T04:34:18.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.537+0000 7f55d2d19640 1 --2- 172.21.15.37:0/2766004509 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f55b8078bc0 0x7f55b807b0d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.537+0000 7f55d2d19640 1 --2- 172.21.15.37:0/2766004509 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55cc0cafa0 0x7f55cc15d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.537+0000 7f55d2d19640 1 --2- 172.21.15.37:0/2766004509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55cc0af7a0 0x7f55cc151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.537+0000 7f55d2d19640 1 --2- 172.21.15.37:0/2766004509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55cc070010 0x7f55cc09cec0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.538+0000 7f55d2d19640 1 -- 172.21.15.37:0/2766004509 >> 172.21.15.37:0/2766004509 conn(0x7f55cc06d3f0 msgr2=0x7f55cc15b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:18.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.538+0000 7f55d2d19640 1 -- 172.21.15.37:0/2766004509 shutdown_connections 2024-09-17T04:34:18.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.538+0000 7f55d2d19640 1 -- 172.21.15.37:0/2766004509 wait complete. 2024-09-17T04:34:18.549 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:34:18.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.666+0000 7f5d1d069640 1 -- 172.21.15.37:0/1181046233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d180d1610 msgr2=0x7f5d180d23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:18.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.666+0000 7f5d1d069640 1 --2- 172.21.15.37:0/1181046233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d180d1610 0x7f5d180d23b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009820 tx=0x7f5d0c02a5a0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.666+0000 7f5d1d069640 1 -- 172.21.15.37:0/1181046233 shutdown_connections 2024-09-17T04:34:18.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.666+0000 7f5d1d069640 1 --2- 172.21.15.37:0/1181046233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d180d1610 0x7f5d180d23b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.666+0000 7f5d1d069640 1 -- 172.21.15.37:0/1181046233 >> 172.21.15.37:0/1181046233 conn(0x7f5d1808bef0 msgr2=0x7f5d180d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:18.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.666+0000 7f5d1d069640 1 -- 172.21.15.37:0/1181046233 shutdown_connections 2024-09-17T04:34:18.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.666+0000 7f5d1d069640 1 -- 172.21.15.37:0/1181046233 wait complete. 2024-09-17T04:34:18.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.668+0000 7f5d1d069640 1 Processor -- start 2024-09-17T04:34:18.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.668+0000 7f5d1d069640 1 -- start start 2024-09-17T04:34:18.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.669+0000 7f5d1d069640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d1806f9f0 0x7f5d181e88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:18.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.669+0000 7f5d1d069640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5d180b3430 0x7f5d180b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:18.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.669+0000 7f5d1d069640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d180b61a0 0x7f5d180b65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:18.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.669+0000 7f5d1d069640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5d180d3000 con 0x7f5d1806f9f0 2024-09-17T04:34:18.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.669+0000 7f5d1d069640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5d180b3b90 con 0x7f5d180b61a0 2024-09-17T04:34:18.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.669+0000 7f5d1d069640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f5d180d3180 con 0x7f5d180b3430 2024-09-17T04:34:18.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.669+0000 7f5d16d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d1806f9f0 0x7f5d181e88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:18.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.669+0000 7f5d16d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d1806f9f0 0x7f5d181e88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38482/0 (socket says 172.21.15.37:38482) 2024-09-17T04:34:18.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.669+0000 7f5d16d9d640 1 -- 172.21.15.37:0/314347196 learned_addr learned my addr 172.21.15.37:0/314347196 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:18.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.669+0000 7f5d1759e640 1 --2- 172.21.15.37:0/314347196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d180b61a0 0x7f5d180b65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:18.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.669+0000 7f5d1659c640 1 --2- 172.21.15.37:0/314347196 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5d180b3430 0x7f5d180b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:18.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.670+0000 7f5d16d9d640 1 -- 172.21.15.37:0/314347196 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5d180b3430 msgr2=0x7f5d180b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:18.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.670+0000 7f5d16d9d640 1 --2- 172.21.15.37:0/314347196 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5d180b3430 0x7f5d180b5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.670+0000 7f5d16d9d640 1 -- 172.21.15.37:0/314347196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d180b61a0 msgr2=0x7f5d180b65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:18.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.670+0000 7f5d16d9d640 1 --2- 172.21.15.37:0/314347196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d180b61a0 0x7f5d180b65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.670+0000 7f5d16d9d640 1 -- 172.21.15.37:0/314347196 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d181f96a0 con 0x7f5d1806f9f0 2024-09-17T04:34:18.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.670+0000 7f5d16d9d640 1 --2- 172.21.15.37:0/314347196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d1806f9f0 0x7f5d181e88d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800c430 tx=0x7f5d0800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:18.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.670+0000 7f5cff7fe640 1 -- 172.21.15.37:0/314347196 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d08017070 con 0x7f5d1806f9f0 2024-09-17T04:34:18.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.670+0000 7f5cff7fe640 1 -- 172.21.15.37:0/314347196 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d080053c0 con 0x7f5d1806f9f0 2024-09-17T04:34:18.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.670+0000 7f5cff7fe640 1 -- 172.21.15.37:0/314347196 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d08003850 con 0x7f5d1806f9f0 2024-09-17T04:34:18.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.670+0000 7f5d1d069640 1 -- 172.21.15.37:0/314347196 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d180b5180 con 0x7f5d1806f9f0 2024-09-17T04:34:18.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.670+0000 7f5d1d069640 1 -- 172.21.15.37:0/314347196 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d1815cf10 con 0x7f5d1806f9f0 2024-09-17T04:34:18.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.671+0000 7f5d1d069640 1 -- 172.21.15.37:0/314347196 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d180d0f00 con 0x7f5d1806f9f0 2024-09-17T04:34:18.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.672+0000 7f5cff7fe640 1 -- 172.21.15.37:0/314347196 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5d0801a070 con 0x7f5d1806f9f0 2024-09-17T04:34:18.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.672+0000 7f5cff7fe640 1 --2- 172.21.15.37:0/314347196 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f5d04078bc0 0x7f5d0407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:18.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.672+0000 7f5cff7fe640 1 -- 172.21.15.37:0/314347196 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6248+0+0 (secure 0 0 0) 0x7f5d0809a220 con 0x7f5d1806f9f0 2024-09-17T04:34:18.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.673+0000 7f5d1659c640 1 --2- 172.21.15.37:0/314347196 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f5d04078bc0 0x7f5d0407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:18.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.673+0000 7f5d1659c640 1 --2- 172.21.15.37:0/314347196 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f5d04078bc0 0x7f5d0407b0d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c034000 tx=0x7f5d0c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:18.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.677+0000 7f5cff7fe640 1 -- 172.21.15.37:0/314347196 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f5d08061df0 con 0x7f5d1806f9f0 2024-09-17T04:34:18.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.864+0000 7f5d1d069640 1 -- 172.21.15.37:0/314347196 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d1806a150 con 0x7f5d1806f9f0 2024-09-17T04:34:18.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.865+0000 7f5cff7fe640 1 -- 172.21.15.37:0/314347196 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+18717 (secure 0 0 0) 0x7f5d08066820 con 0x7f5d1806f9f0 2024-09-17T04:34:18.866 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:18.866 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":40,"fsid":"a908a603-430c-4209-8294-0c3460b35f42","created":"2024-09-17T04:32:54.139221+0000","modified":"2024-09-17T04:34:17.928819+0000","last_up_change":"2024-09-17T04:33:06.922210+0000","last_in_change":"2024-09-17T04:32:57.754118+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:33:08.461907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:33:57.558905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:58.446650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"recovery_fs_metadata","create_time":"2024-09-17T04:34:16.436885+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ea4b8d1-0d22-4dd9-b913-9df585ef24ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6809","nonce":2283587593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6811","nonce":2283587593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6815","nonce":2283587593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6813","nonce":2283587593}]},"public_addr":"172.21.15.37:6809/2283587593","cluster_addr":"172.21.15.37:6811/2283587593","heartbeat_back_addr":"172.21.15.37:6815/2283587593","heartbeat_front_addr":"172.21.15.37:6813/2283587593","state":["exists","up"]},{"osd":1,"uuid":"ac745a33-e8ad-4083-9c18-71bf81982b76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6825","nonce":2657955935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6827","nonce":2657955935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6831","nonce":2657955935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6829","nonce":2657955935}]},"public_addr":"172.21.15.37:6825/2657955935","cluster_addr":"172.21.15.37:6827/2657955935","heartbeat_back_addr":"172.21.15.37:6831/2657955935","heartbeat_front_addr":"172.21.15.37:6829/2657955935","state":["exists","up"]},{"osd":2,"uuid":"3c63f3d8-132a-4735-b0bc-d64313a46503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6817","nonce":579720176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6819","nonce":579720176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6823","nonce":579720176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6821","nonce":579720176}]},"public_addr":"172.21.15.37:6817/579720176","cluster_addr":"172.21.15.37:6819/579720176","heartbeat_back_addr":"172.21.15.37:6823/579720176","heartbeat_front_addr":"172.21.15.37:6821/579720176","state":["exists","up"]},{"osd":3,"uuid":"4f8c6ca2-ea1d-4610-9127-52c4dd8bc8e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6801","nonce":809975826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6803","nonce":809975826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6807","nonce":809975826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6805","nonce":809975826}]},"public_addr":"172.21.15.37:6801/809975826","cluster_addr":"172.21.15.37:6803/809975826","heartbeat_back_addr":"172.21.15.37:6807/809975826","heartbeat_front_addr":"172.21.15.37:6805/809975826","state":["exists","up"]},{"osd":4,"uuid":"63d0c4ab-f115-47b4-9611-12a10153473a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6819","nonce":2509785752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6821","nonce":2509785752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6825","nonce":2509785752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6823","nonce":2509785752}]},"public_addr":"172.21.15.92:6819/2509785752","cluster_addr":"172.21.15.92:6821/2509785752","heartbeat_back_addr":"172.21.15.92:6825/2509785752","heartbeat_front_addr":"172.21.15.92:6823/2509785752","state":["exists","up"]},{"osd":5,"uuid":"4c56c04d-0c07-4f76-9d75-6335121310ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6827","nonce":248263832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6829","nonce":248263832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6833","nonce":248263832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6831","nonce":248263832}]},"public_addr":"172.21.15.92:6827/248263832","cluster_addr":"172.21.15.92:6829/248263832","heartbeat_back_addr":"172.21.15.92:6833/248263832","heartbeat_front_addr":"172.21.15.92:6831/248263832","state":["exists","up"]},{"osd":6,"uuid":"b30eae7c-096a-4d24-ac51-96b8b13ecaee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6811","nonce":4078365985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6813","nonce":4078365985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6817","nonce":4078365985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6815","nonce":4078365985}]},"public_addr":"172.21.15.92:6811/4078365985","cluster_addr":"172.21.15.92:6813/4078365985","heartbeat_back_addr":"172.21.15.92:6817/4078365985","heartbeat_front_addr":"172.21.15.92:6815/4078365985","state":["exists","up"]},{"osd":7,"uuid":"62f12540-d374-4a4c-9ff7-63b7304fe77f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6803","nonce":3444787223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6805","nonce":3444787223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6809","nonce":3444787223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6807","nonce":3444787223}]},"public_addr":"172.21.15.92:6803/3444787223","cluster_addr":"172.21.15.92:6805/3444787223","heartbeat_back_addr":"172.21.15.92:6809/3444787223","heartbeat_front_addr":"172.21.15.92:6807/3444787223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.006420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.111135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.103325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.072656+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.200989+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.060455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.774773+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.865778+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/1358600605":"2024-09-18T04:34:15.699884+0000","172.21.15.37:6833/1358600605":"2024-09-18T04:34:15.699884+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:34:18.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.867+0000 7f5d1d069640 1 -- 172.21.15.37:0/314347196 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f5d04078bc0 msgr2=0x7f5d0407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:18.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.867+0000 7f5d1d069640 1 --2- 172.21.15.37:0/314347196 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f5d04078bc0 0x7f5d0407b0d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c034000 tx=0x7f5d0c004730 comp rx=0 tx=0).stop 2024-09-17T04:34:18.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.867+0000 7f5d1d069640 1 -- 172.21.15.37:0/314347196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d1806f9f0 msgr2=0x7f5d181e88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:18.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.867+0000 7f5d1d069640 1 --2- 172.21.15.37:0/314347196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d1806f9f0 0x7f5d181e88d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800c430 tx=0x7f5d0800c8f0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.868+0000 7f5d1d069640 1 -- 172.21.15.37:0/314347196 shutdown_connections 2024-09-17T04:34:18.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.868+0000 7f5d1d069640 1 --2- 172.21.15.37:0/314347196 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f5d04078bc0 0x7f5d0407b0d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.868+0000 7f5d1d069640 1 --2- 172.21.15.37:0/314347196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d180b61a0 0x7f5d180b65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.868+0000 7f5d1d069640 1 --2- 172.21.15.37:0/314347196 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5d180b3430 0x7f5d180b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.868+0000 7f5d1d069640 1 --2- 172.21.15.37:0/314347196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d1806f9f0 0x7f5d181e88d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:18.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.868+0000 7f5d1d069640 1 -- 172.21.15.37:0/314347196 >> 172.21.15.37:0/314347196 conn(0x7f5d1808bef0 msgr2=0x7f5d181617a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:18.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.868+0000 7f5d1d069640 1 -- 172.21.15.37:0/314347196 shutdown_connections 2024-09-17T04:34:18.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.868+0000 7f5d1d069640 1 -- 172.21.15.37:0/314347196 wait complete. 2024-09-17T04:34:18.879 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set recovery_fs standby_count_wanted 0 2024-09-17T04:34:18.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.999+0000 7f51ddf28640 1 -- 172.21.15.37:0/2495597313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51c0004d40 msgr2=0x7f51c0005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:19.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.999+0000 7f51ddf28640 1 --2- 172.21.15.37:0/2495597313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51c0004d40 0x7f51c0005150 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f51d0002230 tx=0x7f51c0005830 comp rx=0 tx=0).stop 2024-09-17T04:34:19.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.999+0000 7f51d7fff640 1 -- 172.21.15.37:0/2495597313 reap_dead start 2024-09-17T04:34:19.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.999+0000 7f51ddf28640 1 -- 172.21.15.37:0/2495597313 shutdown_connections 2024-09-17T04:34:19.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.999+0000 7f51ddf28640 1 -- 172.21.15.37:0/2495597313 >> 172.21.15.37:0/2495597313 conn(0x7f51d80d2450 msgr2=0x7f51d80d2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:19.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:18.999+0000 7f51ddf28640 1 -- 172.21.15.37:0/2495597313 shutdown_connections 2024-09-17T04:34:19.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.000+0000 7f51ddf28640 1 -- 172.21.15.37:0/2495597313 wait complete. 2024-09-17T04:34:19.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.001+0000 7f51ddf28640 1 Processor -- start 2024-09-17T04:34:19.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.002+0000 7f51ddf28640 1 -- start start 2024-09-17T04:34:19.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.003+0000 7f51ddf28640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51c0004d40 0x7f51d8157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:19.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.003+0000 7f51ddf28640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d806ed50 0x7f51d806f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:19.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.003+0000 7f51ddf28640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f51d81f8b60 0x7f51d81f8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:19.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.003+0000 7f51ddf28640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f51d80cdac0 con 0x7f51d806ed50 2024-09-17T04:34:19.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.003+0000 7f51ddf28640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f51d8160410 con 0x7f51c0004d40 2024-09-17T04:34:19.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.003+0000 7f51ddf28640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f51d80cdc40 con 0x7f51d81f8b60 2024-09-17T04:34:19.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.003+0000 7f51d6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d806ed50 0x7f51d806f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.003+0000 7f51d6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d806ed50 0x7f51d806f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38492/0 (socket says 172.21.15.37:38492) 2024-09-17T04:34:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.003+0000 7f51d6ffd640 1 -- 172.21.15.37:0/911272569 learned_addr learned my addr 172.21.15.37:0/911272569 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.003+0000 7f51d7fff640 1 --2- 172.21.15.37:0/911272569 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f51d81f8b60 0x7f51d81f8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.003+0000 7f51d77fe640 1 --2- 172.21.15.37:0/911272569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51c0004d40 0x7f51d8157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.003+0000 7f51d6ffd640 1 -- 172.21.15.37:0/911272569 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f51d81f8b60 msgr2=0x7f51d81f8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.003+0000 7f51d6ffd640 1 --2- 172.21.15.37:0/911272569 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f51d81f8b60 0x7f51d81f8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.003+0000 7f51d6ffd640 1 -- 172.21.15.37:0/911272569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51c0004d40 msgr2=0x7f51d8157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.003+0000 7f51d6ffd640 1 --2- 172.21.15.37:0/911272569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51c0004d40 0x7f51d8157ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.003+0000 7f51d6ffd640 1 -- 172.21.15.37:0/911272569 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d81f94b0 con 0x7f51d806ed50 2024-09-17T04:34:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.004+0000 7f51d6ffd640 1 --2- 172.21.15.37:0/911272569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d806ed50 0x7f51d806f160 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f51cc011ae0 tx=0x7f51cc011ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.004+0000 7f51bffff640 1 -- 172.21.15.37:0/911272569 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc00fe70 con 0x7f51d806ed50 2024-09-17T04:34:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.004+0000 7f51bffff640 1 -- 172.21.15.37:0/911272569 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51cc004050 con 0x7f51d806ed50 2024-09-17T04:34:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.004+0000 7f51bffff640 1 -- 172.21.15.37:0/911272569 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc0044a0 con 0x7f51d806ed50 2024-09-17T04:34:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.004+0000 7f51ddf28640 1 -- 172.21.15.37:0/911272569 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d8157670 con 0x7f51d806ed50 2024-09-17T04:34:19.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.004+0000 7f51ddf28640 1 -- 172.21.15.37:0/911272569 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d81568b0 con 0x7f51d806ed50 2024-09-17T04:34:19.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.005+0000 7f51ddf28640 1 -- 172.21.15.37:0/911272569 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51d8156340 con 0x7f51d806ed50 2024-09-17T04:34:19.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.006+0000 7f51bffff640 1 -- 172.21.15.37:0/911272569 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f51cc0035f0 con 0x7f51d806ed50 2024-09-17T04:34:19.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.006+0000 7f51bffff640 1 --2- 172.21.15.37:0/911272569 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f51c4078bc0 0x7f51c407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:19.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.006+0000 7f51bffff640 1 -- 172.21.15.37:0/911272569 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6248+0+0 (secure 0 0 0) 0x7f51cc09ba60 con 0x7f51d806ed50 2024-09-17T04:34:19.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.006+0000 7f51d77fe640 1 --2- 172.21.15.37:0/911272569 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f51c4078bc0 0x7f51c407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:19.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.007+0000 7f51d77fe640 1 --2- 172.21.15.37:0/911272569 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f51c4078bc0 0x7f51c407b0d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009cc0 tx=0x7f51c8005f30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:19.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.011+0000 7f51bffff640 1 -- 172.21.15.37:0/911272569 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f51d81f94b0 con 0x7f51d806ed50 2024-09-17T04:34:19.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:19.236+0000 7f51ddf28640 1 -- 172.21.15.37:0/911272569 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "recovery_fs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f51d80cd580 con 0x7f51d806ed50 2024-09-17T04:34:20.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.208+0000 7f51bffff640 1 -- 172.21.15.37:0/911272569 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "recovery_fs", "var": "standby_count_wanted", "val": "0"}]=0 v21) ==== 123+0+0 (secure 0 0 0) 0x7f51cc063810 con 0x7f51d806ed50 2024-09-17T04:34:20.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.212+0000 7f51ddf28640 1 -- 172.21.15.37:0/911272569 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f51c4078bc0 msgr2=0x7f51c407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:20.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.212+0000 7f51ddf28640 1 --2- 172.21.15.37:0/911272569 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f51c4078bc0 0x7f51c407b0d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009cc0 tx=0x7f51c8005f30 comp rx=0 tx=0).stop 2024-09-17T04:34:20.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.212+0000 7f51ddf28640 1 -- 172.21.15.37:0/911272569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d806ed50 msgr2=0x7f51d806f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:20.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.212+0000 7f51ddf28640 1 --2- 172.21.15.37:0/911272569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d806ed50 0x7f51d806f160 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f51cc011ae0 tx=0x7f51cc011ee0 comp rx=0 tx=0).stop 2024-09-17T04:34:20.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.212+0000 7f51ddf28640 1 -- 172.21.15.37:0/911272569 shutdown_connections 2024-09-17T04:34:20.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.212+0000 7f51ddf28640 1 --2- 172.21.15.37:0/911272569 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f51c4078bc0 0x7f51c407b0d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:20.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.212+0000 7f51ddf28640 1 --2- 172.21.15.37:0/911272569 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f51d81f8b60 0x7f51d81f8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:20.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.212+0000 7f51ddf28640 1 --2- 172.21.15.37:0/911272569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d806ed50 0x7f51d806f160 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:20.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.212+0000 7f51ddf28640 1 --2- 172.21.15.37:0/911272569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51c0004d40 0x7f51d8157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:20.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.212+0000 7f51ddf28640 1 -- 172.21.15.37:0/911272569 >> 172.21.15.37:0/911272569 conn(0x7f51d80d2450 msgr2=0x7f51d8155ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:20.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.212+0000 7f51ddf28640 1 -- 172.21.15.37:0/911272569 shutdown_connections 2024-09-17T04:34:20.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.212+0000 7f51ddf28640 1 -- 172.21.15.37:0/911272569 wait complete. 2024-09-17T04:34:20.226 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T04:34:20.226 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set recovery_fs allow_standby_replay false 2024-09-17T04:34:20.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.354+0000 7f8cdf2de640 1 -- 172.21.15.37:0/3745710847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cc4004d40 msgr2=0x7f8cc40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:20.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.354+0000 7f8cdf2de640 1 --2- 172.21.15.37:0/3745710847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cc4004d40 0x7f8cc40051d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8cd80728b0 tx=0x7f8cc4005830 comp rx=0 tx=0).stop 2024-09-17T04:34:20.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.354+0000 7f8cdd87b640 1 -- 172.21.15.37:0/3745710847 reap_dead start 2024-09-17T04:34:20.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.354+0000 7f8cdf2de640 1 -- 172.21.15.37:0/3745710847 shutdown_connections 2024-09-17T04:34:20.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.354+0000 7f8cdf2de640 1 -- 172.21.15.37:0/3745710847 >> 172.21.15.37:0/3745710847 conn(0x7f8cd80d2480 msgr2=0x7f8cd80d2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:20.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.354+0000 7f8cdf2de640 1 -- 172.21.15.37:0/3745710847 shutdown_connections 2024-09-17T04:34:20.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.355+0000 7f8cdf2de640 1 -- 172.21.15.37:0/3745710847 wait complete. 2024-09-17T04:34:20.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.356+0000 7f8cdf2de640 1 Processor -- start 2024-09-17T04:34:20.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.356+0000 7f8cdf2de640 1 -- start start 2024-09-17T04:34:20.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.358+0000 7f8cdf2de640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cc4004d40 0x7f8cd81f8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:20.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.358+0000 7f8cdf2de640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd81f9420 0x7f8cd809ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:20.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.358+0000 7f8cdf2de640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8cd809f290 0x7f8cd809cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:20.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.358+0000 7f8cdf2de640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd80b7f80 con 0x7f8cc4004d40 2024-09-17T04:34:20.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.358+0000 7f8cdf2de640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd80b7e00 con 0x7f8cd81f9420 2024-09-17T04:34:20.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.358+0000 7f8cdf2de640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd80d0c70 con 0x7f8cd809f290 2024-09-17T04:34:20.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.358+0000 7f8cdd07a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cc4004d40 0x7f8cd81f8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:20.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.358+0000 7f8cdd07a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cc4004d40 0x7f8cd81f8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38522/0 (socket says 172.21.15.37:38522) 2024-09-17T04:34:20.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.358+0000 7f8cdd07a640 1 -- 172.21.15.37:0/279279919 learned_addr learned my addr 172.21.15.37:0/279279919 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:20.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.358+0000 7f8cdc879640 1 --2- 172.21.15.37:0/279279919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd81f9420 0x7f8cd809ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:20.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.358+0000 7f8cdd87b640 1 --2- 172.21.15.37:0/279279919 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8cd809f290 0x7f8cd809cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:20.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.358+0000 7f8cdd07a640 1 -- 172.21.15.37:0/279279919 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8cd809f290 msgr2=0x7f8cd809cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:20.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.358+0000 7f8cdd07a640 1 --2- 172.21.15.37:0/279279919 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8cd809f290 0x7f8cd809cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:20.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.358+0000 7f8cdd07a640 1 -- 172.21.15.37:0/279279919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd81f9420 msgr2=0x7f8cd809ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:20.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.358+0000 7f8cdd07a640 1 --2- 172.21.15.37:0/279279919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd81f9420 0x7f8cd809ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:20.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.358+0000 7f8cdd07a640 1 -- 172.21.15.37:0/279279919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd806e310 con 0x7f8cc4004d40 2024-09-17T04:34:20.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.358+0000 7f8cdd87b640 1 --2- 172.21.15.37:0/279279919 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8cd809f290 0x7f8cd809cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:20.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.358+0000 7f8cdc879640 1 --2- 172.21.15.37:0/279279919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd81f9420 0x7f8cd809ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:20.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.358+0000 7f8cdd07a640 1 --2- 172.21.15.37:0/279279919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cc4004d40 0x7f8cd81f8ee0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4011a50 tx=0x7f8cd4011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:20.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.358+0000 7f8ccdffb640 1 -- 172.21.15.37:0/279279919 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4004460 con 0x7f8cc4004d40 2024-09-17T04:34:20.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.359+0000 7f8ccdffb640 1 -- 172.21.15.37:0/279279919 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8cd4016070 con 0x7f8cc4004d40 2024-09-17T04:34:20.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.359+0000 7f8cdf2de640 1 -- 172.21.15.37:0/279279919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd809fb10 con 0x7f8cc4004d40 2024-09-17T04:34:20.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.359+0000 7f8ccdffb640 1 -- 172.21.15.37:0/279279919 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd400fd10 con 0x7f8cc4004d40 2024-09-17T04:34:20.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.359+0000 7f8cdf2de640 1 -- 172.21.15.37:0/279279919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd81e64b0 con 0x7f8cc4004d40 2024-09-17T04:34:20.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.360+0000 7f8cdf2de640 1 -- 172.21.15.37:0/279279919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cd80d1200 con 0x7f8cc4004d40 2024-09-17T04:34:20.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.361+0000 7f8ccdffb640 1 -- 172.21.15.37:0/279279919 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8cd4018030 con 0x7f8cc4004d40 2024-09-17T04:34:20.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.361+0000 7f8ccdffb640 1 --2- 172.21.15.37:0/279279919 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8cbc078b90 0x7f8cbc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:20.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.361+0000 7f8ccdffb640 1 -- 172.21.15.37:0/279279919 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7f8cd409bff0 con 0x7f8cc4004d40 2024-09-17T04:34:20.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.361+0000 7f8cdc879640 1 --2- 172.21.15.37:0/279279919 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8cbc078b90 0x7f8cbc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:20.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.362+0000 7f8cdc879640 1 --2- 172.21.15.37:0/279279919 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8cbc078b90 0x7f8cbc07b0a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8cd80700b0 tx=0x7f8cc8004040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:20.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.365+0000 7f8ccdffb640 1 -- 172.21.15.37:0/279279919 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8cd4064380 con 0x7f8cc4004d40 2024-09-17T04:34:20.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:20.589+0000 7f8cdf2de640 1 -- 172.21.15.37:0/279279919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "recovery_fs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f8cd80b6570 con 0x7f8cc4004d40 2024-09-17T04:34:21.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.215+0000 7f8ccdffb640 1 -- 172.21.15.37:0/279279919 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "recovery_fs", "var": "allow_standby_replay", "val": "false"}]=0 v22) ==== 127+0+0 (secure 0 0 0) 0x7f8cd4063c80 con 0x7f8cc4004d40 2024-09-17T04:34:21.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.218+0000 7f8cdf2de640 1 -- 172.21.15.37:0/279279919 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8cbc078b90 msgr2=0x7f8cbc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:21.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.218+0000 7f8cdf2de640 1 --2- 172.21.15.37:0/279279919 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8cbc078b90 0x7f8cbc07b0a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8cd80700b0 tx=0x7f8cc8004040 comp rx=0 tx=0).stop 2024-09-17T04:34:21.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.218+0000 7f8cdf2de640 1 -- 172.21.15.37:0/279279919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cc4004d40 msgr2=0x7f8cd81f8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:21.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.218+0000 7f8cdf2de640 1 --2- 172.21.15.37:0/279279919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cc4004d40 0x7f8cd81f8ee0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4011a50 tx=0x7f8cd4011f10 comp rx=0 tx=0).stop 2024-09-17T04:34:21.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.218+0000 7f8cdf2de640 1 -- 172.21.15.37:0/279279919 shutdown_connections 2024-09-17T04:34:21.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.218+0000 7f8cdf2de640 1 --2- 172.21.15.37:0/279279919 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8cbc078b90 0x7f8cbc07b0a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.218+0000 7f8cdf2de640 1 --2- 172.21.15.37:0/279279919 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8cd809f290 0x7f8cd809cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.218+0000 7f8cdf2de640 1 --2- 172.21.15.37:0/279279919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd81f9420 0x7f8cd809ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.218+0000 7f8cdf2de640 1 --2- 172.21.15.37:0/279279919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cc4004d40 0x7f8cd81f8ee0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.218+0000 7f8cdf2de640 1 -- 172.21.15.37:0/279279919 >> 172.21.15.37:0/279279919 conn(0x7f8cd80d2480 msgr2=0x7f8cd80b4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:21.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.219+0000 7f8cdf2de640 1 -- 172.21.15.37:0/279279919 shutdown_connections 2024-09-17T04:34:21.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.219+0000 7f8cdf2de640 1 -- 172.21.15.37:0/279279919 wait complete. 2024-09-17T04:34:21.233 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'recovery_fs' 2024-09-17T04:34:21.233 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:34:21.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.358+0000 7f072fc3f640 1 -- 172.21.15.37:0/3035332840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0710004d40 msgr2=0x7f0710005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:21.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.358+0000 7f072fc3f640 1 --2- 172.21.15.37:0/3035332840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0710004d40 0x7f0710005150 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f07280b9ff0 tx=0x7f0710005830 comp rx=0 tx=0).stop 2024-09-17T04:34:21.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.358+0000 7f072e1dc640 1 -- 172.21.15.37:0/3035332840 reap_dead start 2024-09-17T04:34:21.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.358+0000 7f072fc3f640 1 -- 172.21.15.37:0/3035332840 shutdown_connections 2024-09-17T04:34:21.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.358+0000 7f072fc3f640 1 -- 172.21.15.37:0/3035332840 >> 172.21.15.37:0/3035332840 conn(0x7f072808bef0 msgr2=0x7f07280d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:21.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.358+0000 7f072fc3f640 1 -- 172.21.15.37:0/3035332840 shutdown_connections 2024-09-17T04:34:21.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.358+0000 7f072fc3f640 1 -- 172.21.15.37:0/3035332840 wait complete. 2024-09-17T04:34:21.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.360+0000 7f072fc3f640 1 Processor -- start 2024-09-17T04:34:21.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.360+0000 7f072fc3f640 1 -- start start 2024-09-17T04:34:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.361+0000 7f072fc3f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0710004d40 0x7f072814d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.361+0000 7f072fc3f640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f072815e450 0x7f07280b5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.361+0000 7f072fc3f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07280b5eb0 0x7f072814a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.361+0000 7f072fc3f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f07280b3b00 con 0x7f07280b5eb0 2024-09-17T04:34:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.361+0000 7f072fc3f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f07280b3980 con 0x7f0710004d40 2024-09-17T04:34:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.361+0000 7f072fc3f640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f0728163290 con 0x7f072815e450 2024-09-17T04:34:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.361+0000 7f072e1dc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07280b5eb0 0x7f072814a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.361+0000 7f072e1dc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07280b5eb0 0x7f072814a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38548/0 (socket says 172.21.15.37:38548) 2024-09-17T04:34:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.361+0000 7f072e1dc640 1 -- 172.21.15.37:0/532850794 learned_addr learned my addr 172.21.15.37:0/532850794 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.361+0000 7f072d1da640 1 --2- 172.21.15.37:0/532850794 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f072815e450 0x7f07280b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.361+0000 7f072d9db640 1 --2- 172.21.15.37:0/532850794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0710004d40 0x7f072814d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.362+0000 7f072e1dc640 1 -- 172.21.15.37:0/532850794 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f072815e450 msgr2=0x7f07280b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.362+0000 7f072e1dc640 1 --2- 172.21.15.37:0/532850794 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f072815e450 0x7f07280b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.362+0000 7f072e1dc640 1 -- 172.21.15.37:0/532850794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0710004d40 msgr2=0x7f072814d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.362+0000 7f072e1dc640 1 --2- 172.21.15.37:0/532850794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0710004d40 0x7f072814d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.362+0000 7f072e1dc640 1 -- 172.21.15.37:0/532850794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072814d7c0 con 0x7f07280b5eb0 2024-09-17T04:34:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.362+0000 7f072e1dc640 1 --2- 172.21.15.37:0/532850794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07280b5eb0 0x7f072814a820 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f072002b4d0 tx=0x7f0720033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:21.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.362+0000 7f07267fc640 1 -- 172.21.15.37:0/532850794 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0720042070 con 0x7f07280b5eb0 2024-09-17T04:34:21.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.362+0000 7f07267fc640 1 -- 172.21.15.37:0/532850794 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07200337d0 con 0x7f07280b5eb0 2024-09-17T04:34:21.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.362+0000 7f072fc3f640 1 -- 172.21.15.37:0/532850794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07280b62c0 con 0x7f07280b5eb0 2024-09-17T04:34:21.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.362+0000 7f07267fc640 1 -- 172.21.15.37:0/532850794 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0720038040 con 0x7f07280b5eb0 2024-09-17T04:34:21.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.362+0000 7f072fc3f640 1 -- 172.21.15.37:0/532850794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f072809e6a0 con 0x7f07280b5eb0 2024-09-17T04:34:21.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.363+0000 7f072fc3f640 1 -- 172.21.15.37:0/532850794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f072815a620 con 0x7f07280b5eb0 2024-09-17T04:34:21.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.364+0000 7f07267fc640 1 -- 172.21.15.37:0/532850794 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0720045070 con 0x7f07280b5eb0 2024-09-17T04:34:21.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.365+0000 7f07267fc640 1 --2- 172.21.15.37:0/532850794 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f0714078b90 0x7f071407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:21.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.365+0000 7f07267fc640 1 -- 172.21.15.37:0/532850794 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7f07200baba0 con 0x7f07280b5eb0 2024-09-17T04:34:21.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.365+0000 7f072d9db640 1 --2- 172.21.15.37:0/532850794 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f0714078b90 0x7f071407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:21.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.365+0000 7f072d9db640 1 --2- 172.21.15.37:0/532850794 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f0714078b90 0x7f071407b0a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0718005fd0 tx=0x7f0718009500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:21.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.369+0000 7f07267fc640 1 -- 172.21.15.37:0/532850794 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f07280b62c0 con 0x7f07280b5eb0 2024-09-17T04:34:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.594+0000 7f072fc3f640 1 -- 172.21.15.37:0/532850794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f072806a150 con 0x7f07280b5eb0 2024-09-17T04:34:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.595+0000 7f07267fc640 1 -- 172.21.15.37:0/532850794 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+6507 (secure 0 0 0) 0x7f0720082800 con 0x7f07280b5eb0 2024-09-17T04:34:21.595 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:21.595 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":22,"btime":"2024-09-17T04:34:21:209946+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/199192833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":199192833},{"type":"v1","addr":"172.21.15.37:6833","nonce":199192833}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4832,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/4098913371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4098913371},{"type":"v1","addr":"172.21.15.92:6836","nonce":4098913371}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/383658044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":383658044},{"type":"v1","addr":"172.21.15.92:6837","nonce":383658044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5208,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/4051839052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":4051839052},{"type":"v1","addr":"172.21.15.37:6835","nonce":4051839052}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:59.490265+0000","modified":"2024-09-17T04:34:15.700096+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2},{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:34:17.928780+0000","modified":"2024-09-17T04:34:20.590578+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[5],"metadata_pool":6,"enabled":true,"fs_name":"recovery_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-09-17T04:34:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 22 2024-09-17T04:34:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.597+0000 7f072fc3f640 1 -- 172.21.15.37:0/532850794 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f0714078b90 msgr2=0x7f071407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.597+0000 7f072fc3f640 1 --2- 172.21.15.37:0/532850794 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f0714078b90 0x7f071407b0a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0718005fd0 tx=0x7f0718009500 comp rx=0 tx=0).stop 2024-09-17T04:34:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.597+0000 7f072fc3f640 1 -- 172.21.15.37:0/532850794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07280b5eb0 msgr2=0x7f072814a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.597+0000 7f072fc3f640 1 --2- 172.21.15.37:0/532850794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07280b5eb0 0x7f072814a820 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f072002b4d0 tx=0x7f0720033670 comp rx=0 tx=0).stop 2024-09-17T04:34:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.597+0000 7f072fc3f640 1 -- 172.21.15.37:0/532850794 shutdown_connections 2024-09-17T04:34:21.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.597+0000 7f072fc3f640 1 --2- 172.21.15.37:0/532850794 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f0714078b90 0x7f071407b0a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.597+0000 7f072fc3f640 1 --2- 172.21.15.37:0/532850794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07280b5eb0 0x7f072814a820 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.597+0000 7f072fc3f640 1 --2- 172.21.15.37:0/532850794 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f072815e450 0x7f07280b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.597+0000 7f072fc3f640 1 --2- 172.21.15.37:0/532850794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0710004d40 0x7f072814d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.597+0000 7f072fc3f640 1 -- 172.21.15.37:0/532850794 >> 172.21.15.37:0/532850794 conn(0x7f072808bef0 msgr2=0x7f07280b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:21.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.597+0000 7f072fc3f640 1 -- 172.21.15.37:0/532850794 shutdown_connections 2024-09-17T04:34:21.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.597+0000 7f072fc3f640 1 -- 172.21.15.37:0/532850794 wait complete. 2024-09-17T04:34:21.608 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:34:21.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.726+0000 7fb01f84d640 1 -- 172.21.15.37:0/1426145769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb01806f9f0 msgr2=0x7fb0180d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:21.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.726+0000 7fb01f84d640 1 --2- 172.21.15.37:0/1426145769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb01806f9f0 0x7fb0180d0f00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb0180cb6b0 tx=0x7fb01402a560 comp rx=0 tx=0).stop 2024-09-17T04:34:21.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.726+0000 7fb01f84d640 1 -- 172.21.15.37:0/1426145769 shutdown_connections 2024-09-17T04:34:21.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.726+0000 7fb01f84d640 1 --2- 172.21.15.37:0/1426145769 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb018162640 0x7fb018162a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.726+0000 7fb01f84d640 1 --2- 172.21.15.37:0/1426145769 >> v2:172.21.15.92:3300/0 conn(0x7fb0180b3430 0x7fb0180d1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.726+0000 7fb01f84d640 1 --2- 172.21.15.37:0/1426145769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb01806f9f0 0x7fb0180d0f00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.726+0000 7fb01f84d640 1 -- 172.21.15.37:0/1426145769 >> 172.21.15.37:0/1426145769 conn(0x7fb01808bef0 msgr2=0x7fb0180d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:21.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.726+0000 7fb01f84d640 1 -- 172.21.15.37:0/1426145769 shutdown_connections 2024-09-17T04:34:21.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.726+0000 7fb01f84d640 1 -- 172.21.15.37:0/1426145769 wait complete. 2024-09-17T04:34:21.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.728+0000 7fb01f84d640 1 Processor -- start 2024-09-17T04:34:21.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.728+0000 7fb01f84d640 1 -- start start 2024-09-17T04:34:21.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.729+0000 7fb01f84d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb01806f9f0 0x7fb0181eaf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:21.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.730+0000 7fb01f84d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0180b3430 0x7fb0181e9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:21.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.730+0000 7fb01f84d640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb018162640 0x7fb0181e9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:21.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.730+0000 7fb01f84d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb0180b3b00 con 0x7fb01806f9f0 2024-09-17T04:34:21.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.730+0000 7fb01f84d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb0180b3980 con 0x7fb0180b3430 2024-09-17T04:34:21.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.730+0000 7fb01f84d640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fb018162f90 con 0x7fb018162640 2024-09-17T04:34:21.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.730+0000 7fb01d5e9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb01806f9f0 0x7fb0181eaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:21.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.730+0000 7fb01d5e9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb01806f9f0 0x7fb0181eaf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38556/0 (socket says 172.21.15.37:38556) 2024-09-17T04:34:21.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.730+0000 7fb01d5e9640 1 -- 172.21.15.37:0/3803446488 learned_addr learned my addr 172.21.15.37:0/3803446488 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:21.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.730+0000 7fb01cde8640 1 --2- 172.21.15.37:0/3803446488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0180b3430 0x7fb0181e9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:21.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.730+0000 7fb01ddea640 1 --2- 172.21.15.37:0/3803446488 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb018162640 0x7fb0181e9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:21.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.730+0000 7fb01d5e9640 1 -- 172.21.15.37:0/3803446488 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb018162640 msgr2=0x7fb0181e9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:21.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.730+0000 7fb01d5e9640 1 --2- 172.21.15.37:0/3803446488 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb018162640 0x7fb0181e9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.730+0000 7fb01d5e9640 1 -- 172.21.15.37:0/3803446488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0180b3430 msgr2=0x7fb0181e9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:21.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.730+0000 7fb01d5e9640 1 --2- 172.21.15.37:0/3803446488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0180b3430 0x7fb0181e9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.730+0000 7fb01d5e9640 1 -- 172.21.15.37:0/3803446488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01809e4e0 con 0x7fb01806f9f0 2024-09-17T04:34:21.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.730+0000 7fb01ddea640 1 --2- 172.21.15.37:0/3803446488 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb018162640 0x7fb0181e9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:21.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.730+0000 7fb01d5e9640 1 --2- 172.21.15.37:0/3803446488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb01806f9f0 0x7fb0181eaf70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb014034040 tx=0x7fb014003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:21.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.730+0000 7fb00dffb640 1 -- 172.21.15.37:0/3803446488 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01403e030 con 0x7fb01806f9f0 2024-09-17T04:34:21.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.730+0000 7fb00dffb640 1 -- 172.21.15.37:0/3803446488 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb014003e00 con 0x7fb01806f9f0 2024-09-17T04:34:21.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.730+0000 7fb01f84d640 1 -- 172.21.15.37:0/3803446488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0181eb4b0 con 0x7fb01806f9f0 2024-09-17T04:34:21.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.730+0000 7fb00dffb640 1 -- 172.21.15.37:0/3803446488 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014035040 con 0x7fb01806f9f0 2024-09-17T04:34:21.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.731+0000 7fb01f84d640 1 -- 172.21.15.37:0/3803446488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb01809eaa0 con 0x7fb01806f9f0 2024-09-17T04:34:21.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.732+0000 7fb01f84d640 1 -- 172.21.15.37:0/3803446488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb018163390 con 0x7fb01806f9f0 2024-09-17T04:34:21.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.733+0000 7fb00dffb640 1 -- 172.21.15.37:0/3803446488 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb014041030 con 0x7fb01806f9f0 2024-09-17T04:34:21.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.733+0000 7fb00dffb640 1 --2- 172.21.15.37:0/3803446488 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7faffc078b90 0x7faffc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:21.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.733+0000 7fb00dffb640 1 -- 172.21.15.37:0/3803446488 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7fb0140bb7c0 con 0x7fb01806f9f0 2024-09-17T04:34:21.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.733+0000 7fb01cde8640 1 --2- 172.21.15.37:0/3803446488 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7faffc078b90 0x7faffc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:21.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.734+0000 7fb01cde8640 1 --2- 172.21.15.37:0/3803446488 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7faffc078b90 0x7faffc07b0a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb008005fd0 tx=0x7fb008005de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:21.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.738+0000 7fb00dffb640 1 -- 172.21.15.37:0/3803446488 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb014083a70 con 0x7fb01806f9f0 2024-09-17T04:34:21.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.924+0000 7fb01f84d640 1 -- 172.21.15.37:0/3803446488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb01806a150 con 0x7fb01806f9f0 2024-09-17T04:34:21.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.925+0000 7fb00dffb640 1 -- 172.21.15.37:0/3803446488 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+18704 (secure 0 0 0) 0x7fb014083370 con 0x7fb01806f9f0 2024-09-17T04:34:21.925 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:21.926 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":41,"fsid":"a908a603-430c-4209-8294-0c3460b35f42","created":"2024-09-17T04:32:54.139221+0000","modified":"2024-09-17T04:34:19.179872+0000","last_up_change":"2024-09-17T04:33:06.922210+0000","last_in_change":"2024-09-17T04:32:57.754118+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:33:08.461907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:33:57.558905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:58.446650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"recovery_fs_metadata","create_time":"2024-09-17T04:34:16.436885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ea4b8d1-0d22-4dd9-b913-9df585ef24ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6809","nonce":2283587593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6811","nonce":2283587593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6815","nonce":2283587593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6813","nonce":2283587593}]},"public_addr":"172.21.15.37:6809/2283587593","cluster_addr":"172.21.15.37:6811/2283587593","heartbeat_back_addr":"172.21.15.37:6815/2283587593","heartbeat_front_addr":"172.21.15.37:6813/2283587593","state":["exists","up"]},{"osd":1,"uuid":"ac745a33-e8ad-4083-9c18-71bf81982b76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6825","nonce":2657955935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6827","nonce":2657955935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6831","nonce":2657955935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6829","nonce":2657955935}]},"public_addr":"172.21.15.37:6825/2657955935","cluster_addr":"172.21.15.37:6827/2657955935","heartbeat_back_addr":"172.21.15.37:6831/2657955935","heartbeat_front_addr":"172.21.15.37:6829/2657955935","state":["exists","up"]},{"osd":2,"uuid":"3c63f3d8-132a-4735-b0bc-d64313a46503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6817","nonce":579720176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6819","nonce":579720176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6823","nonce":579720176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6821","nonce":579720176}]},"public_addr":"172.21.15.37:6817/579720176","cluster_addr":"172.21.15.37:6819/579720176","heartbeat_back_addr":"172.21.15.37:6823/579720176","heartbeat_front_addr":"172.21.15.37:6821/579720176","state":["exists","up"]},{"osd":3,"uuid":"4f8c6ca2-ea1d-4610-9127-52c4dd8bc8e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6801","nonce":809975826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6803","nonce":809975826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6807","nonce":809975826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6805","nonce":809975826}]},"public_addr":"172.21.15.37:6801/809975826","cluster_addr":"172.21.15.37:6803/809975826","heartbeat_back_addr":"172.21.15.37:6807/809975826","heartbeat_front_addr":"172.21.15.37:6805/809975826","state":["exists","up"]},{"osd":4,"uuid":"63d0c4ab-f115-47b4-9611-12a10153473a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6819","nonce":2509785752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6821","nonce":2509785752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6825","nonce":2509785752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6823","nonce":2509785752}]},"public_addr":"172.21.15.92:6819/2509785752","cluster_addr":"172.21.15.92:6821/2509785752","heartbeat_back_addr":"172.21.15.92:6825/2509785752","heartbeat_front_addr":"172.21.15.92:6823/2509785752","state":["exists","up"]},{"osd":5,"uuid":"4c56c04d-0c07-4f76-9d75-6335121310ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6827","nonce":248263832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6829","nonce":248263832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6833","nonce":248263832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6831","nonce":248263832}]},"public_addr":"172.21.15.92:6827/248263832","cluster_addr":"172.21.15.92:6829/248263832","heartbeat_back_addr":"172.21.15.92:6833/248263832","heartbeat_front_addr":"172.21.15.92:6831/248263832","state":["exists","up"]},{"osd":6,"uuid":"b30eae7c-096a-4d24-ac51-96b8b13ecaee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6811","nonce":4078365985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6813","nonce":4078365985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6817","nonce":4078365985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6815","nonce":4078365985}]},"public_addr":"172.21.15.92:6811/4078365985","cluster_addr":"172.21.15.92:6813/4078365985","heartbeat_back_addr":"172.21.15.92:6817/4078365985","heartbeat_front_addr":"172.21.15.92:6815/4078365985","state":["exists","up"]},{"osd":7,"uuid":"62f12540-d374-4a4c-9ff7-63b7304fe77f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6803","nonce":3444787223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6805","nonce":3444787223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6809","nonce":3444787223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6807","nonce":3444787223}]},"public_addr":"172.21.15.92:6803/3444787223","cluster_addr":"172.21.15.92:6805/3444787223","heartbeat_back_addr":"172.21.15.92:6809/3444787223","heartbeat_front_addr":"172.21.15.92:6807/3444787223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.006420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.111135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.103325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.072656+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.200989+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.060455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.774773+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.865778+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/1358600605":"2024-09-18T04:34:15.699884+0000","172.21.15.37:6833/1358600605":"2024-09-18T04:34:15.699884+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:34:21.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.928+0000 7fb01f84d640 1 -- 172.21.15.37:0/3803446488 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7faffc078b90 msgr2=0x7faffc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:21.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.928+0000 7fb01f84d640 1 --2- 172.21.15.37:0/3803446488 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7faffc078b90 0x7faffc07b0a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb008005fd0 tx=0x7fb008005de0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.928+0000 7fb01f84d640 1 -- 172.21.15.37:0/3803446488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb01806f9f0 msgr2=0x7fb0181eaf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:21.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.928+0000 7fb01f84d640 1 --2- 172.21.15.37:0/3803446488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb01806f9f0 0x7fb0181eaf70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb014034040 tx=0x7fb014003910 comp rx=0 tx=0).stop 2024-09-17T04:34:21.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.928+0000 7fb01f84d640 1 -- 172.21.15.37:0/3803446488 shutdown_connections 2024-09-17T04:34:21.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.928+0000 7fb01f84d640 1 --2- 172.21.15.37:0/3803446488 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7faffc078b90 0x7faffc07b0a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.928+0000 7fb01f84d640 1 --2- 172.21.15.37:0/3803446488 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb018162640 0x7fb0181e9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.928+0000 7fb01f84d640 1 --2- 172.21.15.37:0/3803446488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0180b3430 0x7fb0181e9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.928+0000 7fb01f84d640 1 --2- 172.21.15.37:0/3803446488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb01806f9f0 0x7fb0181eaf70 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:21.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.928+0000 7fb01f84d640 1 -- 172.21.15.37:0/3803446488 >> 172.21.15.37:0/3803446488 conn(0x7fb01808bef0 msgr2=0x7fb0180b7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:21.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.928+0000 7fb01f84d640 1 -- 172.21.15.37:0/3803446488 shutdown_connections 2024-09-17T04:34:21.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:21.928+0000 7fb01f84d640 1 -- 172.21.15.37:0/3803446488 wait complete. 2024-09-17T04:34:21.939 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T04:34:21.940 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:34:22.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.057+0000 7f3a5d630640 1 -- 172.21.15.37:0/412966711 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3a580d1610 msgr2=0x7f3a580d23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:22.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.057+0000 7f3a5d630640 1 --2- 172.21.15.37:0/412966711 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3a580d1610 0x7f3a580d23b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009820 tx=0x7f3a4c02a510 comp rx=0 tx=0).stop 2024-09-17T04:34:22.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.057+0000 7f3a5d630640 1 -- 172.21.15.37:0/412966711 shutdown_connections 2024-09-17T04:34:22.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.057+0000 7f3a5d630640 1 --2- 172.21.15.37:0/412966711 >> v2:172.21.15.92:3300/0 conn(0x7f3a580b3430 0x7f3a580d28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.057+0000 7f3a5d630640 1 --2- 172.21.15.37:0/412966711 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3a580d1610 0x7f3a580d23b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.057+0000 7f3a5d630640 1 -- 172.21.15.37:0/412966711 >> 172.21.15.37:0/412966711 conn(0x7f3a5808bef0 msgr2=0x7f3a580d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:22.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.057+0000 7f3a5d630640 1 -- 172.21.15.37:0/412966711 shutdown_connections 2024-09-17T04:34:22.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.057+0000 7f3a5d630640 1 -- 172.21.15.37:0/412966711 wait complete. 2024-09-17T04:34:22.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.059+0000 7f3a5d630640 1 Processor -- start 2024-09-17T04:34:22.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.059+0000 7f3a5d630640 1 -- start start 2024-09-17T04:34:22.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.060+0000 7f3a5d630640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3a5806f9f0 0x7f3a580b6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:22.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.060+0000 7f3a5d630640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a580b3430 0x7f3a581f97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:22.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.060+0000 7f3a5d630640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a5814e870 0x7f3a580d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:22.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.060+0000 7f3a5d630640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3a580d2e30 con 0x7f3a5814e870 2024-09-17T04:34:22.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.060+0000 7f3a5d630640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3a580b3b90 con 0x7f3a580b3430 2024-09-17T04:34:22.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.060+0000 7f3a5d630640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f3a580d2fb0 con 0x7f3a5806f9f0 2024-09-17T04:34:22.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.060+0000 7f3a577fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a5814e870 0x7f3a580d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:22.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.060+0000 7f3a577fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a5814e870 0x7f3a580d0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38568/0 (socket says 172.21.15.37:38568) 2024-09-17T04:34:22.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.060+0000 7f3a577fe640 1 -- 172.21.15.37:0/2597955991 learned_addr learned my addr 172.21.15.37:0/2597955991 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:22.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.060+0000 7f3a567fc640 1 --2- 172.21.15.37:0/2597955991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a580b3430 0x7f3a581f97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:22.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.060+0000 7f3a56ffd640 1 --2- 172.21.15.37:0/2597955991 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3a5806f9f0 0x7f3a580b6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:22.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.061+0000 7f3a577fe640 1 -- 172.21.15.37:0/2597955991 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3a5806f9f0 msgr2=0x7f3a580b6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:22.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.061+0000 7f3a577fe640 1 --2- 172.21.15.37:0/2597955991 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3a5806f9f0 0x7f3a580b6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.061+0000 7f3a577fe640 1 -- 172.21.15.37:0/2597955991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a580b3430 msgr2=0x7f3a581f97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:22.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.061+0000 7f3a577fe640 1 --2- 172.21.15.37:0/2597955991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a580b3430 0x7f3a581f97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.061+0000 7f3a577fe640 1 -- 172.21.15.37:0/2597955991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a581f9d00 con 0x7f3a5814e870 2024-09-17T04:34:22.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.061+0000 7f3a577fe640 1 --2- 172.21.15.37:0/2597955991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a5814e870 0x7f3a580d0e60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000efc0 tx=0x7f3a5000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:22.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.061+0000 7f3a3f7fe640 1 -- 172.21.15.37:0/2597955991 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5001b070 con 0x7f3a5814e870 2024-09-17T04:34:22.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.061+0000 7f3a3f7fe640 1 -- 172.21.15.37:0/2597955991 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a50003b80 con 0x7f3a5814e870 2024-09-17T04:34:22.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.061+0000 7f3a5d630640 1 -- 172.21.15.37:0/2597955991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a580cb6e0 con 0x7f3a5814e870 2024-09-17T04:34:22.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.061+0000 7f3a3f7fe640 1 -- 172.21.15.37:0/2597955991 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a50004030 con 0x7f3a5814e870 2024-09-17T04:34:22.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.061+0000 7f3a5d630640 1 -- 172.21.15.37:0/2597955991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a580cd410 con 0x7f3a5814e870 2024-09-17T04:34:22.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.062+0000 7f3a5d630640 1 -- 172.21.15.37:0/2597955991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a580d34d0 con 0x7f3a5814e870 2024-09-17T04:34:22.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.063+0000 7f3a3f7fe640 1 -- 172.21.15.37:0/2597955991 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3a5001e070 con 0x7f3a5814e870 2024-09-17T04:34:22.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.064+0000 7f3a3f7fe640 1 --2- 172.21.15.37:0/2597955991 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3a44078b90 0x7f3a4407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:22.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.064+0000 7f3a3f7fe640 1 -- 172.21.15.37:0/2597955991 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7f3a5009b7f0 con 0x7f3a5814e870 2024-09-17T04:34:22.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.064+0000 7f3a56ffd640 1 --2- 172.21.15.37:0/2597955991 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3a44078b90 0x7f3a4407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:22.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.064+0000 7f3a56ffd640 1 --2- 172.21.15.37:0/2597955991 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3a44078b90 0x7f3a4407b0a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3a48000f60 tx=0x7f3a48009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:22.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.068+0000 7f3a3f7fe640 1 -- 172.21.15.37:0/2597955991 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3a581f9d00 con 0x7f3a5814e870 2024-09-17T04:34:22.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.254+0000 7f3a5d630640 1 -- 172.21.15.37:0/2597955991 --> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3a580b6f70 con 0x7f3a44078b90 2024-09-17T04:34:22.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.260+0000 7f3a3f7fe640 1 -- 172.21.15.37:0/2597955991 <== mgr.4100 v2:172.21.15.92:6800/864337835 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+327445 (secure 0 0 0) 0x7f3a48001420 con 0x7f3a44078b90 2024-09-17T04:34:22.264 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:22.266 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-17T04:34:22.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.264+0000 7f3a5d630640 1 -- 172.21.15.37:0/2597955991 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3a44078b90 msgr2=0x7f3a4407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:22.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.264+0000 7f3a5d630640 1 --2- 172.21.15.37:0/2597955991 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3a44078b90 0x7f3a4407b0a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3a48000f60 tx=0x7f3a48009040 comp rx=0 tx=0).stop 2024-09-17T04:34:22.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.264+0000 7f3a5d630640 1 -- 172.21.15.37:0/2597955991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a5814e870 msgr2=0x7f3a580d0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:22.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.264+0000 7f3a5d630640 1 --2- 172.21.15.37:0/2597955991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a5814e870 0x7f3a580d0e60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000efc0 tx=0x7f3a5000c800 comp rx=0 tx=0).stop 2024-09-17T04:34:22.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.264+0000 7f3a5d630640 1 -- 172.21.15.37:0/2597955991 shutdown_connections 2024-09-17T04:34:22.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.264+0000 7f3a5d630640 1 --2- 172.21.15.37:0/2597955991 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3a44078b90 0x7f3a4407b0a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.264+0000 7f3a5d630640 1 --2- 172.21.15.37:0/2597955991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a5814e870 0x7f3a580d0e60 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.264+0000 7f3a5d630640 1 --2- 172.21.15.37:0/2597955991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a580b3430 0x7f3a581f97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.264+0000 7f3a5d630640 1 --2- 172.21.15.37:0/2597955991 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3a5806f9f0 0x7f3a580b6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.264+0000 7f3a5d630640 1 -- 172.21.15.37:0/2597955991 >> 172.21.15.37:0/2597955991 conn(0x7f3a5808bef0 msgr2=0x7f3a580d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:22.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.264+0000 7f3a5d630640 1 -- 172.21.15.37:0/2597955991 shutdown_connections 2024-09-17T04:34:22.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.264+0000 7f3a5d630640 1 -- 172.21.15.37:0/2597955991 wait complete. 2024-09-17T04:34:22.275 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-09-17T04:34:20.460384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6881824,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":67,"num_read_kb":55,"num_write":202,"num_write_kb":7812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":660,"ondisk_log_size":660,"up":204,"acting":204,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237276,"kb_used_data":16772,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754737444,"statfs":{"total":773094113280,"available":772851142656,"internally_reserved":0,"allocated":17174528,"data_stored":15123721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14335,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":43,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.256404"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":141,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782864+0000","last_change":"2024-09-17T04:33:09.978742+0000","last_active":"2024-09-17T04:34:16.782864+0000","last_peered":"2024-09-17T04:34:16.782864+0000","last_clean":"2024-09-17T04:34:16.782864+0000","last_became_active":"2024-09-17T04:33:09.977496+0000","last_became_peered":"2024-09-17T04:33:09.977496+0000","last_unstale":"2024-09-17T04:34:16.782864+0000","last_undegraded":"2024-09-17T04:34:16.782864+0000","last_fullsized":"2024-09-17T04:34:16.782864+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:08.930340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:08.930340+0000","last_clean_scrub_stamp":"2024-09-17T04:33:08.930340+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:21:26.566102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782493+0000","last_change":"2024-09-17T04:33:59.116928+0000","last_active":"2024-09-17T04:34:16.782493+0000","last_peered":"2024-09-17T04:34:16.782493+0000","last_clean":"2024-09-17T04:34:16.782493+0000","last_became_active":"2024-09-17T04:33:59.115764+0000","last_became_peered":"2024-09-17T04:33:59.115764+0000","last_unstale":"2024-09-17T04:34:16.782493+0000","last_undegraded":"2024-09-17T04:34:16.782493+0000","last_fullsized":"2024-09-17T04:34:16.782493+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:08.489333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"33'9","reported_seq":32,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782875+0000","last_change":"2024-09-17T04:34:00.136520+0000","last_active":"2024-09-17T04:34:16.782875+0000","last_peered":"2024-09-17T04:34:16.782875+0000","last_clean":"2024-09-17T04:34:16.782875+0000","last_became_active":"2024-09-17T04:34:00.136200+0000","last_became_peered":"2024-09-17T04:34:00.136200+0000","last_unstale":"2024-09-17T04:34:16.782875+0000","last_undegraded":"2024-09-17T04:34:16.782875+0000","last_fullsized":"2024-09-17T04:34:16.782875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:38:06.615802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:34:17.200691+0000","last_change":"2024-09-17T04:34:17.200691+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.200691+0000","last_undegraded":"2024-09-17T04:34:17.200691+0000","last_fullsized":"2024-09-17T04:34:17.200691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.209006+0000","last_change":"2024-09-17T04:33:59.121607+0000","last_active":"2024-09-17T04:34:16.209006+0000","last_peered":"2024-09-17T04:34:16.209006+0000","last_clean":"2024-09-17T04:34:16.209006+0000","last_became_active":"2024-09-17T04:33:59.121262+0000","last_became_peered":"2024-09-17T04:33:59.121262+0000","last_unstale":"2024-09-17T04:34:16.209006+0000","last_undegraded":"2024-09-17T04:34:16.209006+0000","last_fullsized":"2024-09-17T04:34:16.209006+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:06.764372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"33'5","reported_seq":29,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-17T04:34:17.217442+0000","last_change":"2024-09-17T04:34:00.146380+0000","last_active":"2024-09-17T04:34:17.217442+0000","last_peered":"2024-09-17T04:34:17.217442+0000","last_clean":"2024-09-17T04:34:17.217442+0000","last_became_active":"2024-09-17T04:34:00.146024+0000","last_became_peered":"2024-09-17T04:34:00.146024+0000","last_unstale":"2024-09-17T04:34:17.217442+0000","last_undegraded":"2024-09-17T04:34:17.217442+0000","last_fullsized":"2024-09-17T04:34:17.217442+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:35:51.784535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-17T04:34:17.215957+0000","last_change":"2024-09-17T04:34:17.215957+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.215957+0000","last_undegraded":"2024-09-17T04:34:17.215957+0000","last_fullsized":"2024-09-17T04:34:17.215957+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-17T04:34:17.217771+0000","last_change":"2024-09-17T04:33:59.109854+0000","last_active":"2024-09-17T04:34:17.217771+0000","last_peered":"2024-09-17T04:34:17.217771+0000","last_clean":"2024-09-17T04:34:17.217771+0000","last_became_active":"2024-09-17T04:33:59.109534+0000","last_became_peered":"2024-09-17T04:33:59.109534+0000","last_unstale":"2024-09-17T04:34:17.217771+0000","last_undegraded":"2024-09-17T04:34:17.217771+0000","last_fullsized":"2024-09-17T04:34:17.217771+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:24.369490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"33'8","reported_seq":28,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.209291+0000","last_change":"2024-09-17T04:34:00.146323+0000","last_active":"2024-09-17T04:34:16.209291+0000","last_peered":"2024-09-17T04:34:16.209291+0000","last_clean":"2024-09-17T04:34:16.209291+0000","last_became_active":"2024-09-17T04:34:00.145997+0000","last_became_peered":"2024-09-17T04:34:00.145997+0000","last_unstale":"2024-09-17T04:34:16.209291+0000","last_undegraded":"2024-09-17T04:34:16.209291+0000","last_fullsized":"2024-09-17T04:34:16.209291+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:41.305573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:34:17.200691+0000","last_change":"2024-09-17T04:34:17.200691+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.200691+0000","last_undegraded":"2024-09-17T04:34:17.200691+0000","last_fullsized":"2024-09-17T04:34:17.200691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.783121+0000","last_change":"2024-09-17T04:33:59.119769+0000","last_active":"2024-09-17T04:34:16.783121+0000","last_peered":"2024-09-17T04:34:16.783121+0000","last_clean":"2024-09-17T04:34:16.783121+0000","last_became_active":"2024-09-17T04:33:59.119378+0000","last_became_peered":"2024-09-17T04:33:59.119378+0000","last_unstale":"2024-09-17T04:34:16.783121+0000","last_undegraded":"2024-09-17T04:34:16.783121+0000","last_fullsized":"2024-09-17T04:34:16.783121+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:02.777362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"33'7","reported_seq":30,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782469+0000","last_change":"2024-09-17T04:34:00.133293+0000","last_active":"2024-09-17T04:34:16.782469+0000","last_peered":"2024-09-17T04:34:16.782469+0000","last_clean":"2024-09-17T04:34:16.782469+0000","last_became_active":"2024-09-17T04:34:00.132910+0000","last_became_peered":"2024-09-17T04:34:00.132910+0000","last_unstale":"2024-09-17T04:34:16.782469+0000","last_undegraded":"2024-09-17T04:34:16.782469+0000","last_fullsized":"2024-09-17T04:34:16.782469+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:03.455957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:34:17.200691+0000","last_change":"2024-09-17T04:34:17.200691+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.200691+0000","last_undegraded":"2024-09-17T04:34:17.200691+0000","last_fullsized":"2024-09-17T04:34:17.200691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782723+0000","last_change":"2024-09-17T04:33:59.114304+0000","last_active":"2024-09-17T04:34:16.782723+0000","last_peered":"2024-09-17T04:34:16.782723+0000","last_clean":"2024-09-17T04:34:16.782723+0000","last_became_active":"2024-09-17T04:33:59.113930+0000","last_became_peered":"2024-09-17T04:33:59.113930+0000","last_unstale":"2024-09-17T04:34:16.782723+0000","last_undegraded":"2024-09-17T04:34:16.782723+0000","last_fullsized":"2024-09-17T04:34:16.782723+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:11.433911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"33'8","reported_seq":29,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782571+0000","last_change":"2024-09-17T04:34:00.135067+0000","last_active":"2024-09-17T04:34:16.782571+0000","last_peered":"2024-09-17T04:34:16.782571+0000","last_clean":"2024-09-17T04:34:16.782571+0000","last_became_active":"2024-09-17T04:34:00.134742+0000","last_became_peered":"2024-09-17T04:34:00.134742+0000","last_unstale":"2024-09-17T04:34:16.782571+0000","last_undegraded":"2024-09-17T04:34:16.782571+0000","last_fullsized":"2024-09-17T04:34:16.782571+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:55.679577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:34:17.200691+0000","last_change":"2024-09-17T04:34:17.200691+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.200691+0000","last_undegraded":"2024-09-17T04:34:17.200691+0000","last_fullsized":"2024-09-17T04:34:17.200691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782787+0000","last_change":"2024-09-17T04:33:59.119426+0000","last_active":"2024-09-17T04:34:16.782787+0000","last_peered":"2024-09-17T04:34:16.782787+0000","last_clean":"2024-09-17T04:34:16.782787+0000","last_became_active":"2024-09-17T04:33:59.118894+0000","last_became_peered":"2024-09-17T04:33:59.118894+0000","last_unstale":"2024-09-17T04:34:16.782787+0000","last_undegraded":"2024-09-17T04:34:16.782787+0000","last_fullsized":"2024-09-17T04:34:16.782787+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:31.864330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"33'5","reported_seq":28,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782356+0000","last_change":"2024-09-17T04:34:00.144557+0000","last_active":"2024-09-17T04:34:16.782356+0000","last_peered":"2024-09-17T04:34:16.782356+0000","last_clean":"2024-09-17T04:34:16.782356+0000","last_became_active":"2024-09-17T04:34:00.144230+0000","last_became_peered":"2024-09-17T04:34:00.144230+0000","last_unstale":"2024-09-17T04:34:16.782356+0000","last_undegraded":"2024-09-17T04:34:16.782356+0000","last_fullsized":"2024-09-17T04:34:16.782356+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:46:25.341350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:34:17.200691+0000","last_change":"2024-09-17T04:34:17.200691+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.200691+0000","last_undegraded":"2024-09-17T04:34:17.200691+0000","last_fullsized":"2024-09-17T04:34:17.200691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"34'5","reported_seq":30,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782694+0000","last_change":"2024-09-17T04:33:59.117234+0000","last_active":"2024-09-17T04:34:16.782694+0000","last_peered":"2024-09-17T04:34:16.782694+0000","last_clean":"2024-09-17T04:34:16.782694+0000","last_became_active":"2024-09-17T04:33:59.116890+0000","last_became_peered":"2024-09-17T04:33:59.116890+0000","last_unstale":"2024-09-17T04:34:16.782694+0000","last_undegraded":"2024-09-17T04:34:16.782694+0000","last_fullsized":"2024-09-17T04:34:16.782694+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:28.423942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"33'8","reported_seq":29,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.208218+0000","last_change":"2024-09-17T04:34:00.145064+0000","last_active":"2024-09-17T04:34:16.208218+0000","last_peered":"2024-09-17T04:34:16.208218+0000","last_clean":"2024-09-17T04:34:16.208218+0000","last_became_active":"2024-09-17T04:34:00.144786+0000","last_became_peered":"2024-09-17T04:34:00.144786+0000","last_unstale":"2024-09-17T04:34:16.208218+0000","last_undegraded":"2024-09-17T04:34:16.208218+0000","last_fullsized":"2024-09-17T04:34:16.208218+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:58.534910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:34:17.200691+0000","last_change":"2024-09-17T04:34:17.200691+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.200691+0000","last_undegraded":"2024-09-17T04:34:17.200691+0000","last_fullsized":"2024-09-17T04:34:17.200691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"34'3","reported_seq":26,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.783093+0000","last_change":"2024-09-17T04:33:59.116203+0000","last_active":"2024-09-17T04:34:16.783093+0000","last_peered":"2024-09-17T04:34:16.783093+0000","last_clean":"2024-09-17T04:34:16.783093+0000","last_became_active":"2024-09-17T04:33:59.115805+0000","last_became_peered":"2024-09-17T04:33:59.115805+0000","last_unstale":"2024-09-17T04:34:16.783093+0000","last_undegraded":"2024-09-17T04:34:16.783093+0000","last_fullsized":"2024-09-17T04:34:16.783093+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:23:51.532043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"33'7","reported_seq":28,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.202809+0000","last_change":"2024-09-17T04:34:00.139613+0000","last_active":"2024-09-17T04:34:16.202809+0000","last_peered":"2024-09-17T04:34:16.202809+0000","last_clean":"2024-09-17T04:34:16.202809+0000","last_became_active":"2024-09-17T04:34:00.139292+0000","last_became_peered":"2024-09-17T04:34:00.139292+0000","last_unstale":"2024-09-17T04:34:16.202809+0000","last_undegraded":"2024-09-17T04:34:16.202809+0000","last_fullsized":"2024-09-17T04:34:16.202809+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:16:46.909065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:34:17.200691+0000","last_change":"2024-09-17T04:34:17.200691+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.200691+0000","last_undegraded":"2024-09-17T04:34:17.200691+0000","last_fullsized":"2024-09-17T04:34:17.200691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.207635+0000","last_change":"2024-09-17T04:33:59.113577+0000","last_active":"2024-09-17T04:34:16.207635+0000","last_peered":"2024-09-17T04:34:16.207635+0000","last_clean":"2024-09-17T04:34:16.207635+0000","last_became_active":"2024-09-17T04:33:59.113210+0000","last_became_peered":"2024-09-17T04:33:59.113210+0000","last_unstale":"2024-09-17T04:34:16.207635+0000","last_undegraded":"2024-09-17T04:34:16.207635+0000","last_fullsized":"2024-09-17T04:34:16.207635+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:06.764372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"33'2","reported_seq":25,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782640+0000","last_change":"2024-09-17T04:34:00.134940+0000","last_active":"2024-09-17T04:34:16.782640+0000","last_peered":"2024-09-17T04:34:16.782640+0000","last_clean":"2024-09-17T04:34:16.782640+0000","last_became_active":"2024-09-17T04:34:00.134621+0000","last_became_peered":"2024-09-17T04:34:00.134621+0000","last_unstale":"2024-09-17T04:34:16.782640+0000","last_undegraded":"2024-09-17T04:34:16.782640+0000","last_fullsized":"2024-09-17T04:34:16.782640+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:02.203052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:34:17.200691+0000","last_change":"2024-09-17T04:34:17.200691+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.200691+0000","last_undegraded":"2024-09-17T04:34:17.200691+0000","last_fullsized":"2024-09-17T04:34:17.200691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-17T04:34:17.217475+0000","last_change":"2024-09-17T04:33:59.116189+0000","last_active":"2024-09-17T04:34:17.217475+0000","last_peered":"2024-09-17T04:34:17.217475+0000","last_clean":"2024-09-17T04:34:17.217475+0000","last_became_active":"2024-09-17T04:33:59.115627+0000","last_became_peered":"2024-09-17T04:33:59.115627+0000","last_unstale":"2024-09-17T04:34:17.217475+0000","last_undegraded":"2024-09-17T04:34:17.217475+0000","last_fullsized":"2024-09-17T04:34:17.217475+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:31.619034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"33'9","reported_seq":30,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.202492+0000","last_change":"2024-09-17T04:34:00.133387+0000","last_active":"2024-09-17T04:34:16.202492+0000","last_peered":"2024-09-17T04:34:16.202492+0000","last_clean":"2024-09-17T04:34:16.202492+0000","last_became_active":"2024-09-17T04:34:00.133075+0000","last_became_peered":"2024-09-17T04:34:00.133075+0000","last_unstale":"2024-09-17T04:34:16.202492+0000","last_undegraded":"2024-09-17T04:34:16.202492+0000","last_fullsized":"2024-09-17T04:34:16.202492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:39.142294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:34:17.200691+0000","last_change":"2024-09-17T04:34:17.200691+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.200691+0000","last_undegraded":"2024-09-17T04:34:17.200691+0000","last_fullsized":"2024-09-17T04:34:17.200691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782701+0000","last_change":"2024-09-17T04:33:59.116483+0000","last_active":"2024-09-17T04:34:16.782701+0000","last_peered":"2024-09-17T04:34:16.782701+0000","last_clean":"2024-09-17T04:34:16.782701+0000","last_became_active":"2024-09-17T04:33:59.116144+0000","last_became_peered":"2024-09-17T04:33:59.116144+0000","last_unstale":"2024-09-17T04:34:16.782701+0000","last_undegraded":"2024-09-17T04:34:16.782701+0000","last_fullsized":"2024-09-17T04:34:16.782701+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:40.142964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"33'8","reported_seq":29,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.202980+0000","last_change":"2024-09-17T04:34:00.134391+0000","last_active":"2024-09-17T04:34:16.202980+0000","last_peered":"2024-09-17T04:34:16.202980+0000","last_clean":"2024-09-17T04:34:16.202980+0000","last_became_active":"2024-09-17T04:34:00.134057+0000","last_became_peered":"2024-09-17T04:34:00.134057+0000","last_unstale":"2024-09-17T04:34:16.202980+0000","last_undegraded":"2024-09-17T04:34:16.202980+0000","last_fullsized":"2024-09-17T04:34:16.202980+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:18.435735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:34:17.200691+0000","last_change":"2024-09-17T04:34:17.200691+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.200691+0000","last_undegraded":"2024-09-17T04:34:17.200691+0000","last_fullsized":"2024-09-17T04:34:17.200691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"34'4","reported_seq":29,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782595+0000","last_change":"2024-09-17T04:33:59.116366+0000","last_active":"2024-09-17T04:34:16.782595+0000","last_peered":"2024-09-17T04:34:16.782595+0000","last_clean":"2024-09-17T04:34:16.782595+0000","last_became_active":"2024-09-17T04:33:59.116060+0000","last_became_peered":"2024-09-17T04:33:59.116060+0000","last_unstale":"2024-09-17T04:34:16.782595+0000","last_undegraded":"2024-09-17T04:34:16.782595+0000","last_fullsized":"2024-09-17T04:34:16.782595+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:23:51.532043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"33'5","reported_seq":26,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.202552+0000","last_change":"2024-09-17T04:34:00.137771+0000","last_active":"2024-09-17T04:34:16.202552+0000","last_peered":"2024-09-17T04:34:16.202552+0000","last_clean":"2024-09-17T04:34:16.202552+0000","last_became_active":"2024-09-17T04:34:00.137424+0000","last_became_peered":"2024-09-17T04:34:00.137424+0000","last_unstale":"2024-09-17T04:34:16.202552+0000","last_undegraded":"2024-09-17T04:34:16.202552+0000","last_fullsized":"2024-09-17T04:34:16.202552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:08:40.126582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:34:17.200691+0000","last_change":"2024-09-17T04:34:17.200691+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.200691+0000","last_undegraded":"2024-09-17T04:34:17.200691+0000","last_fullsized":"2024-09-17T04:34:17.200691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":26,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-17T04:34:17.218006+0000","last_change":"2024-09-17T04:33:59.115490+0000","last_active":"2024-09-17T04:34:17.218006+0000","last_peered":"2024-09-17T04:34:17.218006+0000","last_clean":"2024-09-17T04:34:17.218006+0000","last_became_active":"2024-09-17T04:33:59.115105+0000","last_became_peered":"2024-09-17T04:33:59.115105+0000","last_unstale":"2024-09-17T04:34:17.218006+0000","last_undegraded":"2024-09-17T04:34:17.218006+0000","last_fullsized":"2024-09-17T04:34:17.218006+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:25.941127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"33'7","reported_seq":28,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.202804+0000","last_change":"2024-09-17T04:34:00.131812+0000","last_active":"2024-09-17T04:34:16.202804+0000","last_peered":"2024-09-17T04:34:16.202804+0000","last_clean":"2024-09-17T04:34:16.202804+0000","last_became_active":"2024-09-17T04:34:00.131461+0000","last_became_peered":"2024-09-17T04:34:00.131461+0000","last_unstale":"2024-09-17T04:34:16.202804+0000","last_undegraded":"2024-09-17T04:34:16.202804+0000","last_fullsized":"2024-09-17T04:34:16.202804+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:08:13.529442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-17T04:34:17.217017+0000","last_change":"2024-09-17T04:34:17.217017+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.217017+0000","last_undegraded":"2024-09-17T04:34:17.217017+0000","last_fullsized":"2024-09-17T04:34:17.217017+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"34'2","reported_seq":28,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-17T04:34:17.217918+0000","last_change":"2024-09-17T04:33:59.112268+0000","last_active":"2024-09-17T04:34:17.217918+0000","last_peered":"2024-09-17T04:34:17.217918+0000","last_clean":"2024-09-17T04:34:17.217918+0000","last_became_active":"2024-09-17T04:33:59.111822+0000","last_became_peered":"2024-09-17T04:33:59.111822+0000","last_unstale":"2024-09-17T04:34:17.217918+0000","last_undegraded":"2024-09-17T04:34:17.217918+0000","last_fullsized":"2024-09-17T04:34:17.217918+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:21:44.164301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"33'6","reported_seq":27,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782660+0000","last_change":"2024-09-17T04:34:00.138877+0000","last_active":"2024-09-17T04:34:16.782660+0000","last_peered":"2024-09-17T04:34:16.782660+0000","last_clean":"2024-09-17T04:34:16.782660+0000","last_became_active":"2024-09-17T04:34:00.138529+0000","last_became_peered":"2024-09-17T04:34:00.138529+0000","last_unstale":"2024-09-17T04:34:16.782660+0000","last_undegraded":"2024-09-17T04:34:16.782660+0000","last_fullsized":"2024-09-17T04:34:16.782660+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:05:34.986501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:34:17.200691+0000","last_change":"2024-09-17T04:34:17.200691+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.200691+0000","last_undegraded":"2024-09-17T04:34:17.200691+0000","last_fullsized":"2024-09-17T04:34:17.200691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"34'3","reported_seq":29,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-17T04:34:17.218092+0000","last_change":"2024-09-17T04:33:59.109283+0000","last_active":"2024-09-17T04:34:17.218092+0000","last_peered":"2024-09-17T04:34:17.218092+0000","last_clean":"2024-09-17T04:34:17.218092+0000","last_became_active":"2024-09-17T04:33:59.108953+0000","last_became_peered":"2024-09-17T04:33:59.108953+0000","last_unstale":"2024-09-17T04:34:17.218092+0000","last_undegraded":"2024-09-17T04:34:17.218092+0000","last_fullsized":"2024-09-17T04:34:17.218092+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:39.786291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"33'9","reported_seq":30,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.208173+0000","last_change":"2024-09-17T04:34:00.143589+0000","last_active":"2024-09-17T04:34:16.208173+0000","last_peered":"2024-09-17T04:34:16.208173+0000","last_clean":"2024-09-17T04:34:16.208173+0000","last_became_active":"2024-09-17T04:34:00.143288+0000","last_became_peered":"2024-09-17T04:34:00.143288+0000","last_unstale":"2024-09-17T04:34:16.208173+0000","last_undegraded":"2024-09-17T04:34:16.208173+0000","last_fullsized":"2024-09-17T04:34:16.208173+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:43:32.412802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:34:17.200691+0000","last_change":"2024-09-17T04:34:17.200691+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.200691+0000","last_undegraded":"2024-09-17T04:34:17.200691+0000","last_fullsized":"2024-09-17T04:34:17.200691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"34'2","reported_seq":25,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782969+0000","last_change":"2024-09-17T04:33:59.118082+0000","last_active":"2024-09-17T04:34:16.782969+0000","last_peered":"2024-09-17T04:34:16.782969+0000","last_clean":"2024-09-17T04:34:16.782969+0000","last_became_active":"2024-09-17T04:33:59.117749+0000","last_became_peered":"2024-09-17T04:33:59.117749+0000","last_unstale":"2024-09-17T04:34:16.782969+0000","last_undegraded":"2024-09-17T04:34:16.782969+0000","last_fullsized":"2024-09-17T04:34:16.782969+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:22:09.267634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"33'5","reported_seq":29,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-17T04:34:17.217965+0000","last_change":"2024-09-17T04:34:00.140363+0000","last_active":"2024-09-17T04:34:17.217965+0000","last_peered":"2024-09-17T04:34:17.217965+0000","last_clean":"2024-09-17T04:34:17.217965+0000","last_became_active":"2024-09-17T04:34:00.138809+0000","last_became_peered":"2024-09-17T04:34:00.138809+0000","last_unstale":"2024-09-17T04:34:17.217965+0000","last_undegraded":"2024-09-17T04:34:17.217965+0000","last_fullsized":"2024-09-17T04:34:17.217965+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:50.605107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:34:17.200691+0000","last_change":"2024-09-17T04:34:17.200691+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.200691+0000","last_undegraded":"2024-09-17T04:34:17.200691+0000","last_fullsized":"2024-09-17T04:34:17.200691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"34'15","reported_seq":37,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.208743+0000","last_change":"2024-09-17T04:33:59.122217+0000","last_active":"2024-09-17T04:34:16.208743+0000","last_peered":"2024-09-17T04:34:16.208743+0000","last_clean":"2024-09-17T04:34:16.208743+0000","last_became_active":"2024-09-17T04:33:59.121866+0000","last_became_peered":"2024-09-17T04:33:59.121866+0000","last_unstale":"2024-09-17T04:34:16.208743+0000","last_undegraded":"2024-09-17T04:34:16.208743+0000","last_fullsized":"2024-09-17T04:34:16.208743+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:33.592661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"33'6","reported_seq":27,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782699+0000","last_change":"2024-09-17T04:34:00.147713+0000","last_active":"2024-09-17T04:34:16.782699+0000","last_peered":"2024-09-17T04:34:16.782699+0000","last_clean":"2024-09-17T04:34:16.782699+0000","last_became_active":"2024-09-17T04:34:00.147099+0000","last_became_peered":"2024-09-17T04:34:00.147099+0000","last_unstale":"2024-09-17T04:34:16.782699+0000","last_undegraded":"2024-09-17T04:34:16.782699+0000","last_fullsized":"2024-09-17T04:34:16.782699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:22:45.296651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:34:17.200691+0000","last_change":"2024-09-17T04:34:17.200691+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.200691+0000","last_undegraded":"2024-09-17T04:34:17.200691+0000","last_fullsized":"2024-09-17T04:34:17.200691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.208721+0000","last_change":"2024-09-17T04:33:59.120796+0000","last_active":"2024-09-17T04:34:16.208721+0000","last_peered":"2024-09-17T04:34:16.208721+0000","last_clean":"2024-09-17T04:34:16.208721+0000","last_became_active":"2024-09-17T04:33:59.120478+0000","last_became_peered":"2024-09-17T04:33:59.120478+0000","last_unstale":"2024-09-17T04:34:16.208721+0000","last_undegraded":"2024-09-17T04:34:16.208721+0000","last_fullsized":"2024-09-17T04:34:16.208721+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:54.677479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"33'6","reported_seq":29,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782700+0000","last_change":"2024-09-17T04:34:00.148329+0000","last_active":"2024-09-17T04:34:16.782700+0000","last_peered":"2024-09-17T04:34:16.782700+0000","last_clean":"2024-09-17T04:34:16.782700+0000","last_became_active":"2024-09-17T04:34:00.147988+0000","last_became_peered":"2024-09-17T04:34:00.147988+0000","last_unstale":"2024-09-17T04:34:16.782700+0000","last_undegraded":"2024-09-17T04:34:16.782700+0000","last_fullsized":"2024-09-17T04:34:16.782700+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:09:37.721140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:34:17.200691+0000","last_change":"2024-09-17T04:34:17.200691+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.200691+0000","last_undegraded":"2024-09-17T04:34:17.200691+0000","last_fullsized":"2024-09-17T04:34:17.200691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.783012+0000","last_change":"2024-09-17T04:33:59.116224+0000","last_active":"2024-09-17T04:34:16.783012+0000","last_peered":"2024-09-17T04:34:16.783012+0000","last_clean":"2024-09-17T04:34:16.783012+0000","last_became_active":"2024-09-17T04:33:59.115862+0000","last_became_peered":"2024-09-17T04:33:59.115862+0000","last_unstale":"2024-09-17T04:34:16.783012+0000","last_undegraded":"2024-09-17T04:34:16.783012+0000","last_fullsized":"2024-09-17T04:34:16.783012+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:22:13.092980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"33'15","reported_seq":35,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.209172+0000","last_change":"2024-09-17T04:34:00.145327+0000","last_active":"2024-09-17T04:34:16.209172+0000","last_peered":"2024-09-17T04:34:16.209172+0000","last_clean":"2024-09-17T04:34:16.209172+0000","last_became_active":"2024-09-17T04:34:00.142809+0000","last_became_peered":"2024-09-17T04:34:00.142809+0000","last_unstale":"2024-09-17T04:34:16.209172+0000","last_undegraded":"2024-09-17T04:34:16.209172+0000","last_fullsized":"2024-09-17T04:34:16.209172+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:47:41.427853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:34:17.200691+0000","last_change":"2024-09-17T04:34:17.200691+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.200691+0000","last_undegraded":"2024-09-17T04:34:17.200691+0000","last_fullsized":"2024-09-17T04:34:17.200691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"34'5","reported_seq":31,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-17T04:34:17.218175+0000","last_change":"2024-09-17T04:33:59.109143+0000","last_active":"2024-09-17T04:34:17.218175+0000","last_peered":"2024-09-17T04:34:17.218175+0000","last_clean":"2024-09-17T04:34:17.218175+0000","last_became_active":"2024-09-17T04:33:59.108847+0000","last_became_peered":"2024-09-17T04:33:59.108847+0000","last_unstale":"2024-09-17T04:34:17.218175+0000","last_undegraded":"2024-09-17T04:34:17.218175+0000","last_fullsized":"2024-09-17T04:34:17.218175+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:18.019734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"33'10","reported_seq":31,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782887+0000","last_change":"2024-09-17T04:34:00.132724+0000","last_active":"2024-09-17T04:34:16.782887+0000","last_peered":"2024-09-17T04:34:16.782887+0000","last_clean":"2024-09-17T04:34:16.782887+0000","last_became_active":"2024-09-17T04:34:00.131882+0000","last_became_peered":"2024-09-17T04:34:00.131882+0000","last_unstale":"2024-09-17T04:34:16.782887+0000","last_undegraded":"2024-09-17T04:34:16.782887+0000","last_fullsized":"2024-09-17T04:34:16.782887+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:48.008056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:34:17.200691+0000","last_change":"2024-09-17T04:34:17.200691+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.200691+0000","last_undegraded":"2024-09-17T04:34:17.200691+0000","last_fullsized":"2024-09-17T04:34:17.200691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"34'23","reported_seq":45,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.209367+0000","last_change":"2024-09-17T04:33:59.124191+0000","last_active":"2024-09-17T04:34:16.209367+0000","last_peered":"2024-09-17T04:34:16.209367+0000","last_clean":"2024-09-17T04:34:16.209367+0000","last_became_active":"2024-09-17T04:33:59.123823+0000","last_became_peered":"2024-09-17T04:33:59.123823+0000","last_unstale":"2024-09-17T04:34:16.209367+0000","last_undegraded":"2024-09-17T04:34:16.209367+0000","last_fullsized":"2024-09-17T04:34:16.209367+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:39:49.197130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"33'6","reported_seq":27,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.207955+0000","last_change":"2024-09-17T04:34:00.146174+0000","last_active":"2024-09-17T04:34:16.207955+0000","last_peered":"2024-09-17T04:34:16.207955+0000","last_clean":"2024-09-17T04:34:16.207955+0000","last_became_active":"2024-09-17T04:34:00.145752+0000","last_became_peered":"2024-09-17T04:34:00.145752+0000","last_unstale":"2024-09-17T04:34:16.207955+0000","last_undegraded":"2024-09-17T04:34:16.207955+0000","last_fullsized":"2024-09-17T04:34:16.207955+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:44.352160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:34:17.200691+0000","last_change":"2024-09-17T04:34:17.200691+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.200691+0000","last_undegraded":"2024-09-17T04:34:17.200691+0000","last_fullsized":"2024-09-17T04:34:17.200691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"34'5","reported_seq":28,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.208160+0000","last_change":"2024-09-17T04:33:59.112913+0000","last_active":"2024-09-17T04:34:16.208160+0000","last_peered":"2024-09-17T04:34:16.208160+0000","last_clean":"2024-09-17T04:34:16.208160+0000","last_became_active":"2024-09-17T04:33:59.112540+0000","last_became_peered":"2024-09-17T04:33:59.112540+0000","last_unstale":"2024-09-17T04:34:16.208160+0000","last_undegraded":"2024-09-17T04:34:16.208160+0000","last_fullsized":"2024-09-17T04:34:16.208160+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:33.592661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"33'11","reported_seq":31,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.209061+0000","last_change":"2024-09-17T04:34:00.148749+0000","last_active":"2024-09-17T04:34:16.209061+0000","last_peered":"2024-09-17T04:34:16.209061+0000","last_clean":"2024-09-17T04:34:16.209061+0000","last_became_active":"2024-09-17T04:34:00.147951+0000","last_became_peered":"2024-09-17T04:34:00.147951+0000","last_unstale":"2024-09-17T04:34:16.209061+0000","last_undegraded":"2024-09-17T04:34:16.209061+0000","last_fullsized":"2024-09-17T04:34:16.209061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:46.711607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-17T04:34:17.216503+0000","last_change":"2024-09-17T04:34:17.216503+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.216503+0000","last_undegraded":"2024-09-17T04:34:17.216503+0000","last_fullsized":"2024-09-17T04:34:17.216503+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782571+0000","last_change":"2024-09-17T04:33:59.112992+0000","last_active":"2024-09-17T04:34:16.782571+0000","last_peered":"2024-09-17T04:34:16.782571+0000","last_clean":"2024-09-17T04:34:16.782571+0000","last_became_active":"2024-09-17T04:33:59.112657+0000","last_became_peered":"2024-09-17T04:33:59.112657+0000","last_unstale":"2024-09-17T04:34:16.782571+0000","last_undegraded":"2024-09-17T04:34:16.782571+0000","last_fullsized":"2024-09-17T04:34:16.782571+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:06.203780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"33'9","reported_seq":29,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.209632+0000","last_change":"2024-09-17T04:34:00.150547+0000","last_active":"2024-09-17T04:34:16.209632+0000","last_peered":"2024-09-17T04:34:16.209632+0000","last_clean":"2024-09-17T04:34:16.209632+0000","last_became_active":"2024-09-17T04:34:00.146052+0000","last_became_peered":"2024-09-17T04:34:00.146052+0000","last_unstale":"2024-09-17T04:34:16.209632+0000","last_undegraded":"2024-09-17T04:34:16.209632+0000","last_fullsized":"2024-09-17T04:34:16.209632+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:22:12.941844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:34:17.200691+0000","last_change":"2024-09-17T04:34:17.200691+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.200691+0000","last_undegraded":"2024-09-17T04:34:17.200691+0000","last_fullsized":"2024-09-17T04:34:17.200691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.209667+0000","last_change":"2024-09-17T04:33:59.120720+0000","last_active":"2024-09-17T04:34:16.209667+0000","last_peered":"2024-09-17T04:34:16.209667+0000","last_clean":"2024-09-17T04:34:16.209667+0000","last_became_active":"2024-09-17T04:33:59.120386+0000","last_became_peered":"2024-09-17T04:33:59.120386+0000","last_unstale":"2024-09-17T04:34:16.209667+0000","last_undegraded":"2024-09-17T04:34:16.209667+0000","last_fullsized":"2024-09-17T04:34:16.209667+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:43.682967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"33'10","reported_seq":31,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.208017+0000","last_change":"2024-09-17T04:34:00.149518+0000","last_active":"2024-09-17T04:34:16.208017+0000","last_peered":"2024-09-17T04:34:16.208017+0000","last_clean":"2024-09-17T04:34:16.208017+0000","last_became_active":"2024-09-17T04:34:00.149133+0000","last_became_peered":"2024-09-17T04:34:00.149133+0000","last_unstale":"2024-09-17T04:34:16.208017+0000","last_undegraded":"2024-09-17T04:34:16.208017+0000","last_fullsized":"2024-09-17T04:34:16.208017+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:09:37.721140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:34:17.200691+0000","last_change":"2024-09-17T04:34:17.200691+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.200691+0000","last_undegraded":"2024-09-17T04:34:17.200691+0000","last_fullsized":"2024-09-17T04:34:17.200691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-17T04:34:17.217465+0000","last_change":"2024-09-17T04:33:59.109225+0000","last_active":"2024-09-17T04:34:17.217465+0000","last_peered":"2024-09-17T04:34:17.217465+0000","last_clean":"2024-09-17T04:34:17.217465+0000","last_became_active":"2024-09-17T04:33:59.108890+0000","last_became_peered":"2024-09-17T04:33:59.108890+0000","last_unstale":"2024-09-17T04:34:17.217465+0000","last_undegraded":"2024-09-17T04:34:17.217465+0000","last_fullsized":"2024-09-17T04:34:17.217465+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:59.447364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"33'10","reported_seq":31,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782095+0000","last_change":"2024-09-17T04:34:00.137017+0000","last_active":"2024-09-17T04:34:16.782095+0000","last_peered":"2024-09-17T04:34:16.782095+0000","last_clean":"2024-09-17T04:34:16.782095+0000","last_became_active":"2024-09-17T04:34:00.136469+0000","last_became_peered":"2024-09-17T04:34:00.136469+0000","last_unstale":"2024-09-17T04:34:16.782095+0000","last_undegraded":"2024-09-17T04:34:16.782095+0000","last_fullsized":"2024-09-17T04:34:16.782095+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:39:50.199228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-17T04:34:17.216515+0000","last_change":"2024-09-17T04:34:17.216515+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.216515+0000","last_undegraded":"2024-09-17T04:34:17.216515+0000","last_fullsized":"2024-09-17T04:34:17.216515+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.208506+0000","last_change":"2024-09-17T04:33:59.121968+0000","last_active":"2024-09-17T04:34:16.208506+0000","last_peered":"2024-09-17T04:34:16.208506+0000","last_clean":"2024-09-17T04:34:16.208506+0000","last_became_active":"2024-09-17T04:33:59.121152+0000","last_became_peered":"2024-09-17T04:33:59.121152+0000","last_unstale":"2024-09-17T04:34:16.208506+0000","last_undegraded":"2024-09-17T04:34:16.208506+0000","last_fullsized":"2024-09-17T04:34:16.208506+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:02:14.225662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"33'8","reported_seq":32,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-17T04:34:17.217456+0000","last_change":"2024-09-17T04:34:00.132375+0000","last_active":"2024-09-17T04:34:17.217456+0000","last_peered":"2024-09-17T04:34:17.217456+0000","last_clean":"2024-09-17T04:34:17.217456+0000","last_became_active":"2024-09-17T04:34:00.131490+0000","last_became_peered":"2024-09-17T04:34:00.131490+0000","last_unstale":"2024-09-17T04:34:17.217456+0000","last_undegraded":"2024-09-17T04:34:17.217456+0000","last_fullsized":"2024-09-17T04:34:17.217456+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:56.019994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:34:17.200691+0000","last_change":"2024-09-17T04:34:17.200691+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.200691+0000","last_undegraded":"2024-09-17T04:34:17.200691+0000","last_fullsized":"2024-09-17T04:34:17.200691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":27,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-17T04:34:17.217757+0000","last_change":"2024-09-17T04:33:59.117052+0000","last_active":"2024-09-17T04:34:17.217757+0000","last_peered":"2024-09-17T04:34:17.217757+0000","last_clean":"2024-09-17T04:34:17.217757+0000","last_became_active":"2024-09-17T04:33:59.116738+0000","last_became_peered":"2024-09-17T04:33:59.116738+0000","last_unstale":"2024-09-17T04:34:17.217757+0000","last_undegraded":"2024-09-17T04:34:17.217757+0000","last_fullsized":"2024-09-17T04:34:17.217757+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:09.152247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"33'8","reported_seq":29,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.208267+0000","last_change":"2024-09-17T04:34:00.142714+0000","last_active":"2024-09-17T04:34:16.208267+0000","last_peered":"2024-09-17T04:34:16.208267+0000","last_clean":"2024-09-17T04:34:16.208267+0000","last_became_active":"2024-09-17T04:34:00.142287+0000","last_became_peered":"2024-09-17T04:34:00.142287+0000","last_unstale":"2024-09-17T04:34:16.208267+0000","last_undegraded":"2024-09-17T04:34:16.208267+0000","last_fullsized":"2024-09-17T04:34:16.208267+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:49:15.290254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:34:17.200691+0000","last_change":"2024-09-17T04:34:17.200691+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.200691+0000","last_undegraded":"2024-09-17T04:34:17.200691+0000","last_fullsized":"2024-09-17T04:34:17.200691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":27,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-17T04:34:17.218129+0000","last_change":"2024-09-17T04:33:59.114365+0000","last_active":"2024-09-17T04:34:17.218129+0000","last_peered":"2024-09-17T04:34:17.218129+0000","last_clean":"2024-09-17T04:34:17.218129+0000","last_became_active":"2024-09-17T04:33:59.114058+0000","last_became_peered":"2024-09-17T04:33:59.114058+0000","last_unstale":"2024-09-17T04:34:17.218129+0000","last_undegraded":"2024-09-17T04:34:17.218129+0000","last_fullsized":"2024-09-17T04:34:17.218129+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:43:41.290086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"33'7","reported_seq":30,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782457+0000","last_change":"2024-09-17T04:34:00.143488+0000","last_active":"2024-09-17T04:34:16.782457+0000","last_peered":"2024-09-17T04:34:16.782457+0000","last_clean":"2024-09-17T04:34:16.782457+0000","last_became_active":"2024-09-17T04:34:00.143166+0000","last_became_peered":"2024-09-17T04:34:00.143166+0000","last_unstale":"2024-09-17T04:34:16.782457+0000","last_undegraded":"2024-09-17T04:34:16.782457+0000","last_fullsized":"2024-09-17T04:34:16.782457+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:27:05.531961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:34:17.200691+0000","last_change":"2024-09-17T04:34:17.200691+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.200691+0000","last_undegraded":"2024-09-17T04:34:17.200691+0000","last_fullsized":"2024-09-17T04:34:17.200691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-09-17T04:34:17.216472+0000","last_change":"2024-09-17T04:34:17.216472+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.216472+0000","last_undegraded":"2024-09-17T04:34:17.216472+0000","last_fullsized":"2024-09-17T04:34:17.216472+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'3","reported_seq":26,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.202844+0000","last_change":"2024-09-17T04:33:59.119452+0000","last_active":"2024-09-17T04:34:16.202844+0000","last_peered":"2024-09-17T04:34:16.202844+0000","last_clean":"2024-09-17T04:34:16.202844+0000","last_became_active":"2024-09-17T04:33:59.119108+0000","last_became_peered":"2024-09-17T04:33:59.119108+0000","last_unstale":"2024-09-17T04:34:16.202844+0000","last_undegraded":"2024-09-17T04:34:16.202844+0000","last_fullsized":"2024-09-17T04:34:16.202844+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:52.325898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"33'15","reported_seq":36,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782996+0000","last_change":"2024-09-17T04:34:00.136203+0000","last_active":"2024-09-17T04:34:16.782996+0000","last_peered":"2024-09-17T04:34:16.782996+0000","last_clean":"2024-09-17T04:34:16.782996+0000","last_became_active":"2024-09-17T04:34:00.135871+0000","last_became_peered":"2024-09-17T04:34:00.135871+0000","last_unstale":"2024-09-17T04:34:16.782996+0000","last_undegraded":"2024-09-17T04:34:16.782996+0000","last_fullsized":"2024-09-17T04:34:16.782996+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:41.010278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:34:17.200691+0000","last_change":"2024-09-17T04:34:17.200691+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.200691+0000","last_undegraded":"2024-09-17T04:34:17.200691+0000","last_fullsized":"2024-09-17T04:34:17.200691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"34'3","reported_seq":25,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.209174+0000","last_change":"2024-09-17T04:33:59.124080+0000","last_active":"2024-09-17T04:34:16.209174+0000","last_peered":"2024-09-17T04:34:16.209174+0000","last_clean":"2024-09-17T04:34:16.209174+0000","last_became_active":"2024-09-17T04:33:59.123624+0000","last_became_peered":"2024-09-17T04:33:59.123624+0000","last_unstale":"2024-09-17T04:34:16.209174+0000","last_undegraded":"2024-09-17T04:34:16.209174+0000","last_fullsized":"2024-09-17T04:34:16.209174+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:52.054448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"33'9","reported_seq":33,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-17T04:34:17.217885+0000","last_change":"2024-09-17T04:34:00.148658+0000","last_active":"2024-09-17T04:34:17.217885+0000","last_peered":"2024-09-17T04:34:17.217885+0000","last_clean":"2024-09-17T04:34:17.217885+0000","last_became_active":"2024-09-17T04:34:00.148350+0000","last_became_peered":"2024-09-17T04:34:00.148350+0000","last_unstale":"2024-09-17T04:34:17.217885+0000","last_undegraded":"2024-09-17T04:34:17.217885+0000","last_fullsized":"2024-09-17T04:34:17.217885+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:25:19.913461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":26,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782442+0000","last_change":"2024-09-17T04:33:59.113364+0000","last_active":"2024-09-17T04:34:16.782442+0000","last_peered":"2024-09-17T04:34:16.782442+0000","last_clean":"2024-09-17T04:34:16.782442+0000","last_became_active":"2024-09-17T04:33:59.112021+0000","last_became_peered":"2024-09-17T04:33:59.112021+0000","last_unstale":"2024-09-17T04:34:16.782442+0000","last_undegraded":"2024-09-17T04:34:16.782442+0000","last_fullsized":"2024-09-17T04:34:16.782442+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:53:00.824171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"33'6","reported_seq":27,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.783079+0000","last_change":"2024-09-17T04:34:00.143656+0000","last_active":"2024-09-17T04:34:16.783079+0000","last_peered":"2024-09-17T04:34:16.783079+0000","last_clean":"2024-09-17T04:34:16.783079+0000","last_became_active":"2024-09-17T04:34:00.143327+0000","last_became_peered":"2024-09-17T04:34:00.143327+0000","last_unstale":"2024-09-17T04:34:16.783079+0000","last_undegraded":"2024-09-17T04:34:16.783079+0000","last_fullsized":"2024-09-17T04:34:16.783079+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:02:15.227761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:34:17.200691+0000","last_change":"2024-09-17T04:34:17.200691+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.200691+0000","last_undegraded":"2024-09-17T04:34:17.200691+0000","last_fullsized":"2024-09-17T04:34:17.200691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"34'8","reported_seq":31,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.783198+0000","last_change":"2024-09-17T04:33:59.113533+0000","last_active":"2024-09-17T04:34:16.783198+0000","last_peered":"2024-09-17T04:34:16.783198+0000","last_clean":"2024-09-17T04:34:16.783198+0000","last_became_active":"2024-09-17T04:33:59.113062+0000","last_became_peered":"2024-09-17T04:33:59.113062+0000","last_unstale":"2024-09-17T04:34:16.783198+0000","last_undegraded":"2024-09-17T04:34:16.783198+0000","last_fullsized":"2024-09-17T04:34:16.783198+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:00.673571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"33'5","reported_seq":28,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782294+0000","last_change":"2024-09-17T04:34:00.139991+0000","last_active":"2024-09-17T04:34:16.782294+0000","last_peered":"2024-09-17T04:34:16.782294+0000","last_clean":"2024-09-17T04:34:16.782294+0000","last_became_active":"2024-09-17T04:34:00.139640+0000","last_became_peered":"2024-09-17T04:34:00.139640+0000","last_unstale":"2024-09-17T04:34:16.782294+0000","last_undegraded":"2024-09-17T04:34:16.782294+0000","last_fullsized":"2024-09-17T04:34:16.782294+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:57:02.382202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T04:34:17.200691+0000","last_change":"2024-09-17T04:34:17.200691+0000","last_active":"2024-09-17T04:34:17.200691+0000","last_peered":"2024-09-17T04:34:17.200691+0000","last_clean":"2024-09-17T04:34:17.200691+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T04:34:17.200691+0000","last_undegraded":"2024-09-17T04:34:17.200691+0000","last_fullsized":"2024-09-17T04:34:17.200691+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"33'9","reported_seq":32,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782571+0000","last_change":"2024-09-17T04:34:00.148182+0000","last_active":"2024-09-17T04:34:16.782571+0000","last_peered":"2024-09-17T04:34:16.782571+0000","last_clean":"2024-09-17T04:34:16.782571+0000","last_became_active":"2024-09-17T04:34:00.147537+0000","last_became_peered":"2024-09-17T04:34:00.147537+0000","last_unstale":"2024-09-17T04:34:16.782571+0000","last_undegraded":"2024-09-17T04:34:16.782571+0000","last_fullsized":"2024-09-17T04:34:16.782571+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:37:23.029519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"33'12","reported_seq":33,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782580+0000","last_change":"2024-09-17T04:34:00.140013+0000","last_active":"2024-09-17T04:34:16.782580+0000","last_peered":"2024-09-17T04:34:16.782580+0000","last_clean":"2024-09-17T04:34:16.782580+0000","last_became_active":"2024-09-17T04:34:00.139703+0000","last_became_peered":"2024-09-17T04:34:00.139703+0000","last_unstale":"2024-09-17T04:34:16.782580+0000","last_undegraded":"2024-09-17T04:34:16.782580+0000","last_fullsized":"2024-09-17T04:34:16.782580+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:59:37.912219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.22","version":"33'8","reported_seq":31,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782529+0000","last_change":"2024-09-17T04:34:00.131809+0000","last_active":"2024-09-17T04:34:16.782529+0000","last_peered":"2024-09-17T04:34:16.782529+0000","last_clean":"2024-09-17T04:34:16.782529+0000","last_became_active":"2024-09-17T04:34:00.131532+0000","last_became_peered":"2024-09-17T04:34:00.131532+0000","last_unstale":"2024-09-17T04:34:16.782529+0000","last_undegraded":"2024-09-17T04:34:16.782529+0000","last_fullsized":"2024-09-17T04:34:16.782529+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:25:50.372117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.23","version":"33'7","reported_seq":31,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-17T04:34:17.217753+0000","last_change":"2024-09-17T04:34:00.146595+0000","last_active":"2024-09-17T04:34:17.217753+0000","last_peered":"2024-09-17T04:34:17.217753+0000","last_clean":"2024-09-17T04:34:17.217753+0000","last_became_active":"2024-09-17T04:34:00.145812+0000","last_became_peered":"2024-09-17T04:34:00.145812+0000","last_unstale":"2024-09-17T04:34:17.217753+0000","last_undegraded":"2024-09-17T04:34:17.217753+0000","last_fullsized":"2024-09-17T04:34:17.217753+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:15.949787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.24","version":"33'4","reported_seq":25,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.207925+0000","last_change":"2024-09-17T04:34:00.134272+0000","last_active":"2024-09-17T04:34:16.207925+0000","last_peered":"2024-09-17T04:34:16.207925+0000","last_clean":"2024-09-17T04:34:16.207925+0000","last_became_active":"2024-09-17T04:34:00.133409+0000","last_became_peered":"2024-09-17T04:34:00.133409+0000","last_unstale":"2024-09-17T04:34:16.207925+0000","last_undegraded":"2024-09-17T04:34:16.207925+0000","last_fullsized":"2024-09-17T04:34:16.207925+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:38.015410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"33'6","reported_seq":27,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782962+0000","last_change":"2024-09-17T04:34:00.136351+0000","last_active":"2024-09-17T04:34:16.782962+0000","last_peered":"2024-09-17T04:34:16.782962+0000","last_clean":"2024-09-17T04:34:16.782962+0000","last_became_active":"2024-09-17T04:34:00.136007+0000","last_became_peered":"2024-09-17T04:34:00.136007+0000","last_unstale":"2024-09-17T04:34:16.782962+0000","last_undegraded":"2024-09-17T04:34:16.782962+0000","last_fullsized":"2024-09-17T04:34:16.782962+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:37:23.029519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"33'15","reported_seq":35,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.209772+0000","last_change":"2024-09-17T04:34:00.147296+0000","last_active":"2024-09-17T04:34:16.209772+0000","last_peered":"2024-09-17T04:34:16.209772+0000","last_clean":"2024-09-17T04:34:16.209772+0000","last_became_active":"2024-09-17T04:34:00.146984+0000","last_became_peered":"2024-09-17T04:34:00.146984+0000","last_unstale":"2024-09-17T04:34:16.209772+0000","last_undegraded":"2024-09-17T04:34:16.209772+0000","last_fullsized":"2024-09-17T04:34:16.209772+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:51.136426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.27","version":"33'2","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.783051+0000","last_change":"2024-09-17T04:34:00.134911+0000","last_active":"2024-09-17T04:34:16.783051+0000","last_peered":"2024-09-17T04:34:16.783051+0000","last_clean":"2024-09-17T04:34:16.783051+0000","last_became_active":"2024-09-17T04:34:00.133373+0000","last_became_peered":"2024-09-17T04:34:00.133373+0000","last_unstale":"2024-09-17T04:34:16.783051+0000","last_undegraded":"2024-09-17T04:34:16.783051+0000","last_fullsized":"2024-09-17T04:34:16.783051+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:27:05.531961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.28","version":"33'6","reported_seq":27,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.207634+0000","last_change":"2024-09-17T04:34:00.137953+0000","last_active":"2024-09-17T04:34:16.207634+0000","last_peered":"2024-09-17T04:34:16.207634+0000","last_clean":"2024-09-17T04:34:16.207634+0000","last_became_active":"2024-09-17T04:34:00.137577+0000","last_became_peered":"2024-09-17T04:34:00.137577+0000","last_unstale":"2024-09-17T04:34:16.207634+0000","last_undegraded":"2024-09-17T04:34:16.207634+0000","last_fullsized":"2024-09-17T04:34:16.207634+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:33:39.300267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.29","version":"33'8","reported_seq":31,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782112+0000","last_change":"2024-09-17T04:34:00.132040+0000","last_active":"2024-09-17T04:34:16.782112+0000","last_peered":"2024-09-17T04:34:16.782112+0000","last_clean":"2024-09-17T04:34:16.782112+0000","last_became_active":"2024-09-17T04:34:00.131717+0000","last_became_peered":"2024-09-17T04:34:00.131717+0000","last_unstale":"2024-09-17T04:34:16.782112+0000","last_undegraded":"2024-09-17T04:34:16.782112+0000","last_fullsized":"2024-09-17T04:34:16.782112+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:27.558398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"33'9","reported_seq":33,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-17T04:34:17.217617+0000","last_change":"2024-09-17T04:34:00.148488+0000","last_active":"2024-09-17T04:34:17.217617+0000","last_peered":"2024-09-17T04:34:17.217617+0000","last_clean":"2024-09-17T04:34:17.217617+0000","last_became_active":"2024-09-17T04:34:00.148176+0000","last_became_peered":"2024-09-17T04:34:00.148176+0000","last_unstale":"2024-09-17T04:34:17.217617+0000","last_undegraded":"2024-09-17T04:34:17.217617+0000","last_fullsized":"2024-09-17T04:34:17.217617+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:23.552526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2b","version":"33'11","reported_seq":35,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-17T04:34:17.217555+0000","last_change":"2024-09-17T04:34:00.133295+0000","last_active":"2024-09-17T04:34:17.217555+0000","last_peered":"2024-09-17T04:34:17.217555+0000","last_clean":"2024-09-17T04:34:17.217555+0000","last_became_active":"2024-09-17T04:34:00.132965+0000","last_became_peered":"2024-09-17T04:34:00.132965+0000","last_unstale":"2024-09-17T04:34:17.217555+0000","last_undegraded":"2024-09-17T04:34:17.217555+0000","last_fullsized":"2024-09-17T04:34:17.217555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:52.024315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2c","version":"33'8","reported_seq":29,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.202700+0000","last_change":"2024-09-17T04:34:00.140053+0000","last_active":"2024-09-17T04:34:16.202700+0000","last_peered":"2024-09-17T04:34:16.202700+0000","last_clean":"2024-09-17T04:34:16.202700+0000","last_became_active":"2024-09-17T04:34:00.139557+0000","last_became_peered":"2024-09-17T04:34:00.139557+0000","last_unstale":"2024-09-17T04:34:16.202700+0000","last_undegraded":"2024-09-17T04:34:16.202700+0000","last_fullsized":"2024-09-17T04:34:16.202700+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:50:59.831340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2d","version":"33'7","reported_seq":27,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.208822+0000","last_change":"2024-09-17T04:34:00.149546+0000","last_active":"2024-09-17T04:34:16.208822+0000","last_peered":"2024-09-17T04:34:16.208822+0000","last_clean":"2024-09-17T04:34:16.208822+0000","last_became_active":"2024-09-17T04:34:00.149166+0000","last_became_peered":"2024-09-17T04:34:00.149166+0000","last_unstale":"2024-09-17T04:34:16.208822+0000","last_undegraded":"2024-09-17T04:34:16.208822+0000","last_fullsized":"2024-09-17T04:34:16.208822+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:09:46.692534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"33'10","reported_seq":30,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.209585+0000","last_change":"2024-09-17T04:34:00.147773+0000","last_active":"2024-09-17T04:34:16.209585+0000","last_peered":"2024-09-17T04:34:16.209585+0000","last_clean":"2024-09-17T04:34:16.209585+0000","last_became_active":"2024-09-17T04:34:00.147077+0000","last_became_peered":"2024-09-17T04:34:00.147077+0000","last_unstale":"2024-09-17T04:34:16.209585+0000","last_undegraded":"2024-09-17T04:34:16.209585+0000","last_fullsized":"2024-09-17T04:34:16.209585+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:04:29.231102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2f","version":"33'9","reported_seq":32,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.783061+0000","last_change":"2024-09-17T04:34:00.134671+0000","last_active":"2024-09-17T04:34:16.783061+0000","last_peered":"2024-09-17T04:34:16.783061+0000","last_clean":"2024-09-17T04:34:16.783061+0000","last_became_active":"2024-09-17T04:34:00.132323+0000","last_became_peered":"2024-09-17T04:34:00.132323+0000","last_unstale":"2024-09-17T04:34:16.783061+0000","last_undegraded":"2024-09-17T04:34:16.783061+0000","last_fullsized":"2024-09-17T04:34:16.783061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:58.534910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.30","version":"33'9","reported_seq":32,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.783179+0000","last_change":"2024-09-17T04:34:00.147088+0000","last_active":"2024-09-17T04:34:16.783179+0000","last_peered":"2024-09-17T04:34:16.783179+0000","last_clean":"2024-09-17T04:34:16.783179+0000","last_became_active":"2024-09-17T04:34:00.146778+0000","last_became_peered":"2024-09-17T04:34:00.146778+0000","last_unstale":"2024-09-17T04:34:16.783179+0000","last_undegraded":"2024-09-17T04:34:16.783179+0000","last_fullsized":"2024-09-17T04:34:16.783179+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:38.015410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.31","version":"33'7","reported_seq":28,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.207771+0000","last_change":"2024-09-17T04:34:00.135929+0000","last_active":"2024-09-17T04:34:16.207771+0000","last_peered":"2024-09-17T04:34:16.207771+0000","last_clean":"2024-09-17T04:34:16.207771+0000","last_became_active":"2024-09-17T04:34:00.135615+0000","last_became_peered":"2024-09-17T04:34:00.135615+0000","last_unstale":"2024-09-17T04:34:16.207771+0000","last_undegraded":"2024-09-17T04:34:16.207771+0000","last_fullsized":"2024-09-17T04:34:16.207771+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:47:51.639001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.32","version":"33'8","reported_seq":28,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.208760+0000","last_change":"2024-09-17T04:34:00.140479+0000","last_active":"2024-09-17T04:34:16.208760+0000","last_peered":"2024-09-17T04:34:16.208760+0000","last_clean":"2024-09-17T04:34:16.208760+0000","last_became_active":"2024-09-17T04:34:00.140125+0000","last_became_peered":"2024-09-17T04:34:00.140125+0000","last_unstale":"2024-09-17T04:34:16.208760+0000","last_undegraded":"2024-09-17T04:34:16.208760+0000","last_fullsized":"2024-09-17T04:34:16.208760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:05.573033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.33","version":"33'10","reported_seq":31,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.202860+0000","last_change":"2024-09-17T04:34:00.137663+0000","last_active":"2024-09-17T04:34:16.202860+0000","last_peered":"2024-09-17T04:34:16.202860+0000","last_clean":"2024-09-17T04:34:16.202860+0000","last_became_active":"2024-09-17T04:34:00.137306+0000","last_became_peered":"2024-09-17T04:34:00.137306+0000","last_unstale":"2024-09-17T04:34:16.202860+0000","last_undegraded":"2024-09-17T04:34:16.202860+0000","last_fullsized":"2024-09-17T04:34:16.202860+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:24.590166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.34","version":"33'7","reported_seq":28,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.207781+0000","last_change":"2024-09-17T04:34:00.144495+0000","last_active":"2024-09-17T04:34:16.207781+0000","last_peered":"2024-09-17T04:34:16.207781+0000","last_clean":"2024-09-17T04:34:16.207781+0000","last_became_active":"2024-09-17T04:34:00.144027+0000","last_became_peered":"2024-09-17T04:34:00.144027+0000","last_unstale":"2024-09-17T04:34:16.207781+0000","last_undegraded":"2024-09-17T04:34:16.207781+0000","last_fullsized":"2024-09-17T04:34:16.207781+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:09:46.692534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.35","version":"33'6","reported_seq":29,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782980+0000","last_change":"2024-09-17T04:34:00.144759+0000","last_active":"2024-09-17T04:34:16.782980+0000","last_peered":"2024-09-17T04:34:16.782980+0000","last_clean":"2024-09-17T04:34:16.782980+0000","last_became_active":"2024-09-17T04:34:00.144463+0000","last_became_peered":"2024-09-17T04:34:00.144463+0000","last_unstale":"2024-09-17T04:34:16.782980+0000","last_undegraded":"2024-09-17T04:34:16.782980+0000","last_fullsized":"2024-09-17T04:34:16.782980+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:09:46.692534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.36","version":"33'15","reported_seq":36,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782520+0000","last_change":"2024-09-17T04:34:00.147614+0000","last_active":"2024-09-17T04:34:16.782520+0000","last_peered":"2024-09-17T04:34:16.782520+0000","last_clean":"2024-09-17T04:34:16.782520+0000","last_became_active":"2024-09-17T04:34:00.147198+0000","last_became_peered":"2024-09-17T04:34:00.147198+0000","last_unstale":"2024-09-17T04:34:16.782520+0000","last_undegraded":"2024-09-17T04:34:16.782520+0000","last_fullsized":"2024-09-17T04:34:16.782520+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:02.203052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.37","version":"33'11","reported_seq":35,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-17T04:34:17.218052+0000","last_change":"2024-09-17T04:34:00.126244+0000","last_active":"2024-09-17T04:34:17.218052+0000","last_peered":"2024-09-17T04:34:17.218052+0000","last_clean":"2024-09-17T04:34:17.218052+0000","last_became_active":"2024-09-17T04:34:00.125921+0000","last_became_peered":"2024-09-17T04:34:00.125921+0000","last_unstale":"2024-09-17T04:34:17.218052+0000","last_undegraded":"2024-09-17T04:34:17.218052+0000","last_fullsized":"2024-09-17T04:34:17.218052+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:21:47.588781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.38","version":"33'9","reported_seq":32,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782436+0000","last_change":"2024-09-17T04:34:00.136461+0000","last_active":"2024-09-17T04:34:16.782436+0000","last_peered":"2024-09-17T04:34:16.782436+0000","last_clean":"2024-09-17T04:34:16.782436+0000","last_became_active":"2024-09-17T04:34:00.136127+0000","last_became_peered":"2024-09-17T04:34:00.136127+0000","last_unstale":"2024-09-17T04:34:16.782436+0000","last_undegraded":"2024-09-17T04:34:16.782436+0000","last_fullsized":"2024-09-17T04:34:16.782436+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:57:08.760910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"33'5","reported_seq":28,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782769+0000","last_change":"2024-09-17T04:34:00.135963+0000","last_active":"2024-09-17T04:34:16.782769+0000","last_peered":"2024-09-17T04:34:16.782769+0000","last_clean":"2024-09-17T04:34:16.782769+0000","last_became_active":"2024-09-17T04:34:00.135466+0000","last_became_peered":"2024-09-17T04:34:00.135466+0000","last_unstale":"2024-09-17T04:34:16.782769+0000","last_undegraded":"2024-09-17T04:34:16.782769+0000","last_fullsized":"2024-09-17T04:34:16.782769+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:49:15.290254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"33'6","reported_seq":26,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.208577+0000","last_change":"2024-09-17T04:34:00.139638+0000","last_active":"2024-09-17T04:34:16.208577+0000","last_peered":"2024-09-17T04:34:16.208577+0000","last_clean":"2024-09-17T04:34:16.208577+0000","last_became_active":"2024-09-17T04:34:00.139252+0000","last_became_peered":"2024-09-17T04:34:00.139252+0000","last_unstale":"2024-09-17T04:34:16.208577+0000","last_undegraded":"2024-09-17T04:34:16.208577+0000","last_fullsized":"2024-09-17T04:34:16.208577+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:42.085058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3b","version":"33'4","reported_seq":25,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.202697+0000","last_change":"2024-09-17T04:34:00.148227+0000","last_active":"2024-09-17T04:34:16.202697+0000","last_peered":"2024-09-17T04:34:16.202697+0000","last_clean":"2024-09-17T04:34:16.202697+0000","last_became_active":"2024-09-17T04:34:00.147902+0000","last_became_peered":"2024-09-17T04:34:00.147902+0000","last_unstale":"2024-09-17T04:34:16.202697+0000","last_undegraded":"2024-09-17T04:34:16.202697+0000","last_fullsized":"2024-09-17T04:34:16.202697+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:52.789669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3c","version":"33'8","reported_seq":29,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782495+0000","last_change":"2024-09-17T04:34:00.138845+0000","last_active":"2024-09-17T04:34:16.782495+0000","last_peered":"2024-09-17T04:34:16.782495+0000","last_clean":"2024-09-17T04:34:16.782495+0000","last_became_active":"2024-09-17T04:34:00.138517+0000","last_became_peered":"2024-09-17T04:34:00.138517+0000","last_unstale":"2024-09-17T04:34:16.782495+0000","last_undegraded":"2024-09-17T04:34:16.782495+0000","last_fullsized":"2024-09-17T04:34:16.782495+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:17.893360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3d","version":"33'7","reported_seq":28,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.207934+0000","last_change":"2024-09-17T04:34:00.146084+0000","last_active":"2024-09-17T04:34:16.207934+0000","last_peered":"2024-09-17T04:34:16.207934+0000","last_clean":"2024-09-17T04:34:16.207934+0000","last_became_active":"2024-09-17T04:34:00.145314+0000","last_became_peered":"2024-09-17T04:34:00.145314+0000","last_unstale":"2024-09-17T04:34:16.207934+0000","last_undegraded":"2024-09-17T04:34:16.207934+0000","last_fullsized":"2024-09-17T04:34:16.207934+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:25.874838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3e","version":"33'4","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.209542+0000","last_change":"2024-09-17T04:34:00.141408+0000","last_active":"2024-09-17T04:34:16.209542+0000","last_peered":"2024-09-17T04:34:16.209542+0000","last_clean":"2024-09-17T04:34:16.209542+0000","last_became_active":"2024-09-17T04:34:00.141077+0000","last_became_peered":"2024-09-17T04:34:00.141077+0000","last_unstale":"2024-09-17T04:34:16.209542+0000","last_undegraded":"2024-09-17T04:34:16.209542+0000","last_fullsized":"2024-09-17T04:34:16.209542+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:24:35.627644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"33'8","reported_seq":29,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T04:34:16.782936+0000","last_change":"2024-09-17T04:34:00.133489+0000","last_active":"2024-09-17T04:34:16.782936+0000","last_peered":"2024-09-17T04:34:16.782936+0000","last_clean":"2024-09-17T04:34:16.782936+0000","last_became_active":"2024-09-17T04:34:00.133172+0000","last_became_peered":"2024-09-17T04:34:00.133172+0000","last_unstale":"2024-09-17T04:34:16.782936+0000","last_undegraded":"2024-09-17T04:34:16.782936+0000","last_fullsized":"2024-09-17T04:34:16.782936+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:34.594760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":10,"acting":10,"num_store_stats":1},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":504,"ondisk_log_size":504,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":97,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84,"ondisk_log_size":84,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30076,"kb_used_data":3036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341764,"statfs":{"total":96636764160,"available":96605966336,"internally_reserved":0,"allocated":3108864,"data_stored":2853686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542162,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29492,"kb_used_data":2456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342348,"statfs":{"total":96636764160,"available":96606564352,"internally_reserved":0,"allocated":2514944,"data_stored":2263318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542163,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":4516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":4624384,"data_stored":4366465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542162,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31636,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340204,"statfs":{"total":96636764160,"available":96604368896,"internally_reserved":0,"allocated":442368,"data_stored":178926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542163,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":430080,"data_stored":172161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542162,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":430080,"data_stored":172161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":4504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":4612096,"data_stored":4360470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542162,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1011712,"data_stored":756534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:34:22.280 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:34:22.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.433+0000 7f6517152640 1 -- 172.21.15.37:0/1328498089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65100d1610 msgr2=0x7f65100d23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:22.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.433+0000 7f6517152640 1 --2- 172.21.15.37:0/1328498089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65100d1610 0x7f65100d23b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f65100cb6b0 tx=0x7f64f802a510 comp rx=0 tx=0).stop 2024-09-17T04:34:22.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.433+0000 7f6517152640 1 -- 172.21.15.37:0/1328498089 shutdown_connections 2024-09-17T04:34:22.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.433+0000 7f6517152640 1 --2- 172.21.15.37:0/1328498089 >> v2:172.21.15.92:3300/0 conn(0x7f65100b3430 0x7f65100d28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.433+0000 7f6517152640 1 --2- 172.21.15.37:0/1328498089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65100d1610 0x7f65100d23b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.433+0000 7f6517152640 1 -- 172.21.15.37:0/1328498089 >> 172.21.15.37:0/1328498089 conn(0x7f651008bef0 msgr2=0x7f65100d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:22.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.433+0000 7f6517152640 1 -- 172.21.15.37:0/1328498089 shutdown_connections 2024-09-17T04:34:22.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.433+0000 7f6517152640 1 -- 172.21.15.37:0/1328498089 wait complete. 2024-09-17T04:34:22.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.435+0000 7f6517152640 1 Processor -- start 2024-09-17T04:34:22.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.435+0000 7f6517152640 1 -- start start 2024-09-17T04:34:22.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.437+0000 7f6517152640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f651006f9f0 0x7f651015bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:22.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.437+0000 7f6517152640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65100b3430 0x7f65100b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:22.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.437+0000 7f6517152640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f65100b5620 0x7f65100b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:22.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.437+0000 7f6517152640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f65100d2e30 con 0x7f651006f9f0 2024-09-17T04:34:22.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.437+0000 7f6517152640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f65100b3b90 con 0x7f65100b3430 2024-09-17T04:34:22.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.437+0000 7f6517152640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f65100d2fb0 con 0x7f65100b5620 2024-09-17T04:34:22.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.437+0000 7f6514eee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f651006f9f0 0x7f651015bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:22.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.437+0000 7f6514eee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f651006f9f0 0x7f651015bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38586/0 (socket says 172.21.15.37:38586) 2024-09-17T04:34:22.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.437+0000 7f6514eee640 1 -- 172.21.15.37:0/1613751798 learned_addr learned my addr 172.21.15.37:0/1613751798 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:22.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.437+0000 7f650ffff640 1 --2- 172.21.15.37:0/1613751798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65100b3430 0x7f65100b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:22.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.437+0000 7f65156ef640 1 --2- 172.21.15.37:0/1613751798 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f65100b5620 0x7f65100b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:22.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.437+0000 7f6514eee640 1 -- 172.21.15.37:0/1613751798 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f65100b5620 msgr2=0x7f65100b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:22.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.437+0000 7f6514eee640 1 --2- 172.21.15.37:0/1613751798 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f65100b5620 0x7f65100b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.437+0000 7f6514eee640 1 -- 172.21.15.37:0/1613751798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65100b3430 msgr2=0x7f65100b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:22.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.437+0000 7f6514eee640 1 --2- 172.21.15.37:0/1613751798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65100b3430 0x7f65100b50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.437+0000 7f6514eee640 1 -- 172.21.15.37:0/1613751798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f651015c2c0 con 0x7f651006f9f0 2024-09-17T04:34:22.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.437+0000 7f6514eee640 1 --2- 172.21.15.37:0/1613751798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f651006f9f0 0x7f651015bd80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f650000b420 tx=0x7f650000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:22.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.437+0000 7f650d7fa640 1 -- 172.21.15.37:0/1613751798 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6500011830 con 0x7f651006f9f0 2024-09-17T04:34:22.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.438+0000 7f650d7fa640 1 -- 172.21.15.37:0/1613751798 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6500016070 con 0x7f651006f9f0 2024-09-17T04:34:22.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.438+0000 7f6517152640 1 -- 172.21.15.37:0/1613751798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f651015dca0 con 0x7f651006f9f0 2024-09-17T04:34:22.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.438+0000 7f650d7fa640 1 -- 172.21.15.37:0/1613751798 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6500011d60 con 0x7f651006f9f0 2024-09-17T04:34:22.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.438+0000 7f6517152640 1 -- 172.21.15.37:0/1613751798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f651015e260 con 0x7f651006f9f0 2024-09-17T04:34:22.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.439+0000 7f6517152640 1 -- 172.21.15.37:0/1613751798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65100d3460 con 0x7f651006f9f0 2024-09-17T04:34:22.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.439+0000 7f650d7fa640 1 -- 172.21.15.37:0/1613751798 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f650001b460 con 0x7f651006f9f0 2024-09-17T04:34:22.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.440+0000 7f650d7fa640 1 --2- 172.21.15.37:0/1613751798 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f64f0078b90 0x7f64f007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:22.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.440+0000 7f650d7fa640 1 -- 172.21.15.37:0/1613751798 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7f650009a980 con 0x7f651006f9f0 2024-09-17T04:34:22.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.440+0000 7f650ffff640 1 --2- 172.21.15.37:0/1613751798 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f64f0078b90 0x7f64f007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:22.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.441+0000 7f650ffff640 1 --2- 172.21.15.37:0/1613751798 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f64f0078b90 0x7f64f007b0a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f64f8034000 tx=0x7f64f8004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:22.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.445+0000 7f650d7fa640 1 -- 172.21.15.37:0/1613751798 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6500020020 con 0x7f651006f9f0 2024-09-17T04:34:22.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.628+0000 7f6517152640 1 -- 172.21.15.37:0/1613751798 --> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f651006a150 con 0x7f64f0078b90 2024-09-17T04:34:22.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.634+0000 7f650d7fa640 1 -- 172.21.15.37:0/1613751798 <== mgr.4100 v2:172.21.15.92:6800/864337835 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332544 (secure 0 0 0) 0x7f651006a150 con 0x7f64f0078b90 2024-09-17T04:34:22.638 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-17T04:34:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.638+0000 7f6517152640 1 -- 172.21.15.37:0/1613751798 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f64f0078b90 msgr2=0x7f64f007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.638+0000 7f6517152640 1 --2- 172.21.15.37:0/1613751798 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f64f0078b90 0x7f64f007b0a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f64f8034000 tx=0x7f64f8004730 comp rx=0 tx=0).stop 2024-09-17T04:34:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.638+0000 7f6517152640 1 -- 172.21.15.37:0/1613751798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f651006f9f0 msgr2=0x7f651015bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.638+0000 7f6517152640 1 --2- 172.21.15.37:0/1613751798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f651006f9f0 0x7f651015bd80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f650000b420 tx=0x7f650000b8e0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.638+0000 7f6517152640 1 -- 172.21.15.37:0/1613751798 shutdown_connections 2024-09-17T04:34:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.638+0000 7f6517152640 1 --2- 172.21.15.37:0/1613751798 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f64f0078b90 0x7f64f007b0a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.638+0000 7f6517152640 1 --2- 172.21.15.37:0/1613751798 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f65100b5620 0x7f65100b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.638+0000 7f6517152640 1 --2- 172.21.15.37:0/1613751798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65100b3430 0x7f65100b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.638+0000 7f6517152640 1 --2- 172.21.15.37:0/1613751798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f651006f9f0 0x7f651015bd80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.638+0000 7f6517152640 1 -- 172.21.15.37:0/1613751798 >> 172.21.15.37:0/1613751798 conn(0x7f651008bef0 msgr2=0x7f65100d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:22.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.638+0000 7f6517152640 1 -- 172.21.15.37:0/1613751798 shutdown_connections 2024-09-17T04:34:22.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.638+0000 7f6517152640 1 -- 172.21.15.37:0/1613751798 wait complete. 2024-09-17T04:34:22.650 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-09-17T04:34:22.460744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6881824,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":67,"num_read_kb":55,"num_write":202,"num_write_kb":7812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":660,"ondisk_log_size":660,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":255,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237512,"kb_used_data":17096,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754737208,"statfs":{"total":773094113280,"available":772850900992,"internally_reserved":0,"allocated":17506304,"data_stored":15308496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14335,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":43,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001206"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":147,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027649+0000","last_change":"2024-09-17T04:33:09.978742+0000","last_active":"2024-09-17T04:34:20.027649+0000","last_peered":"2024-09-17T04:34:20.027649+0000","last_clean":"2024-09-17T04:34:20.027649+0000","last_became_active":"2024-09-17T04:33:09.977496+0000","last_became_peered":"2024-09-17T04:33:09.977496+0000","last_unstale":"2024-09-17T04:34:20.027649+0000","last_undegraded":"2024-09-17T04:34:20.027649+0000","last_fullsized":"2024-09-17T04:34:20.027649+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:08.930340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:08.930340+0000","last_clean_scrub_stamp":"2024-09-17T04:33:08.930340+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:21:26.566102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027578+0000","last_change":"2024-09-17T04:33:59.116928+0000","last_active":"2024-09-17T04:34:20.027578+0000","last_peered":"2024-09-17T04:34:20.027578+0000","last_clean":"2024-09-17T04:34:20.027578+0000","last_became_active":"2024-09-17T04:33:59.115764+0000","last_became_peered":"2024-09-17T04:33:59.115764+0000","last_unstale":"2024-09-17T04:34:20.027578+0000","last_undegraded":"2024-09-17T04:34:20.027578+0000","last_fullsized":"2024-09-17T04:34:20.027578+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:08.489333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"33'9","reported_seq":39,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.028482+0000","last_change":"2024-09-17T04:34:00.136520+0000","last_active":"2024-09-17T04:34:20.028482+0000","last_peered":"2024-09-17T04:34:20.028482+0000","last_clean":"2024-09-17T04:34:20.028482+0000","last_became_active":"2024-09-17T04:34:00.136200+0000","last_became_peered":"2024-09-17T04:34:00.136200+0000","last_unstale":"2024-09-17T04:34:20.028482+0000","last_undegraded":"2024-09-17T04:34:20.028482+0000","last_fullsized":"2024-09-17T04:34:20.028482+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:07:53.168602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.028624+0000","last_change":"2024-09-17T04:34:18.197084+0000","last_active":"2024-09-17T04:34:20.028624+0000","last_peered":"2024-09-17T04:34:20.028624+0000","last_clean":"2024-09-17T04:34:20.028624+0000","last_became_active":"2024-09-17T04:34:18.196492+0000","last_became_peered":"2024-09-17T04:34:18.196492+0000","last_unstale":"2024-09-17T04:34:20.028624+0000","last_undegraded":"2024-09-17T04:34:20.028624+0000","last_fullsized":"2024-09-17T04:34:20.028624+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:24:40.242567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184361+0000","last_change":"2024-09-17T04:33:59.121607+0000","last_active":"2024-09-17T04:34:19.184361+0000","last_peered":"2024-09-17T04:34:19.184361+0000","last_clean":"2024-09-17T04:34:19.184361+0000","last_became_active":"2024-09-17T04:33:59.121262+0000","last_became_peered":"2024-09-17T04:33:59.121262+0000","last_unstale":"2024-09-17T04:34:19.184361+0000","last_undegraded":"2024-09-17T04:34:19.184361+0000","last_fullsized":"2024-09-17T04:34:19.184361+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:06.764372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"33'5","reported_seq":34,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184271+0000","last_change":"2024-09-17T04:34:00.146380+0000","last_active":"2024-09-17T04:34:19.184271+0000","last_peered":"2024-09-17T04:34:19.184271+0000","last_clean":"2024-09-17T04:34:19.184271+0000","last_became_active":"2024-09-17T04:34:00.146024+0000","last_became_peered":"2024-09-17T04:34:00.146024+0000","last_unstale":"2024-09-17T04:34:19.184271+0000","last_undegraded":"2024-09-17T04:34:19.184271+0000","last_fullsized":"2024-09-17T04:34:19.184271+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:58.430679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.185465+0000","last_change":"2024-09-17T04:34:18.196463+0000","last_active":"2024-09-17T04:34:19.185465+0000","last_peered":"2024-09-17T04:34:19.185465+0000","last_clean":"2024-09-17T04:34:19.185465+0000","last_became_active":"2024-09-17T04:34:18.196118+0000","last_became_peered":"2024-09-17T04:34:18.196118+0000","last_unstale":"2024-09-17T04:34:19.185465+0000","last_undegraded":"2024-09-17T04:34:19.185465+0000","last_fullsized":"2024-09-17T04:34:19.185465+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:09.614698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184519+0000","last_change":"2024-09-17T04:33:59.109854+0000","last_active":"2024-09-17T04:34:19.184519+0000","last_peered":"2024-09-17T04:34:19.184519+0000","last_clean":"2024-09-17T04:34:19.184519+0000","last_became_active":"2024-09-17T04:33:59.109534+0000","last_became_peered":"2024-09-17T04:33:59.109534+0000","last_unstale":"2024-09-17T04:34:19.184519+0000","last_undegraded":"2024-09-17T04:34:19.184519+0000","last_fullsized":"2024-09-17T04:34:19.184519+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:24.369490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"33'8","reported_seq":35,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184239+0000","last_change":"2024-09-17T04:34:00.146323+0000","last_active":"2024-09-17T04:34:19.184239+0000","last_peered":"2024-09-17T04:34:19.184239+0000","last_clean":"2024-09-17T04:34:19.184239+0000","last_became_active":"2024-09-17T04:34:00.145997+0000","last_became_peered":"2024-09-17T04:34:00.145997+0000","last_unstale":"2024-09-17T04:34:19.184239+0000","last_undegraded":"2024-09-17T04:34:19.184239+0000","last_fullsized":"2024-09-17T04:34:19.184239+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:15:33.763016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.028502+0000","last_change":"2024-09-17T04:34:18.200272+0000","last_active":"2024-09-17T04:34:20.028502+0000","last_peered":"2024-09-17T04:34:20.028502+0000","last_clean":"2024-09-17T04:34:20.028502+0000","last_became_active":"2024-09-17T04:34:18.199453+0000","last_became_peered":"2024-09-17T04:34:18.199453+0000","last_unstale":"2024-09-17T04:34:20.028502+0000","last_undegraded":"2024-09-17T04:34:20.028502+0000","last_fullsized":"2024-09-17T04:34:20.028502+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:40.092307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027268+0000","last_change":"2024-09-17T04:33:59.119769+0000","last_active":"2024-09-17T04:34:20.027268+0000","last_peered":"2024-09-17T04:34:20.027268+0000","last_clean":"2024-09-17T04:34:20.027268+0000","last_became_active":"2024-09-17T04:33:59.119378+0000","last_became_peered":"2024-09-17T04:33:59.119378+0000","last_unstale":"2024-09-17T04:34:20.027268+0000","last_undegraded":"2024-09-17T04:34:20.027268+0000","last_fullsized":"2024-09-17T04:34:20.027268+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:02.777362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"33'7","reported_seq":37,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027470+0000","last_change":"2024-09-17T04:34:00.133293+0000","last_active":"2024-09-17T04:34:20.027470+0000","last_peered":"2024-09-17T04:34:20.027470+0000","last_clean":"2024-09-17T04:34:20.027470+0000","last_became_active":"2024-09-17T04:34:00.132910+0000","last_became_peered":"2024-09-17T04:34:00.132910+0000","last_unstale":"2024-09-17T04:34:20.027470+0000","last_undegraded":"2024-09-17T04:34:20.027470+0000","last_fullsized":"2024-09-17T04:34:20.027470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:37:23.029519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.026887+0000","last_change":"2024-09-17T04:34:18.191462+0000","last_active":"2024-09-17T04:34:20.026887+0000","last_peered":"2024-09-17T04:34:20.026887+0000","last_clean":"2024-09-17T04:34:20.026887+0000","last_became_active":"2024-09-17T04:34:18.191162+0000","last_became_peered":"2024-09-17T04:34:18.191162+0000","last_unstale":"2024-09-17T04:34:20.026887+0000","last_undegraded":"2024-09-17T04:34:20.026887+0000","last_fullsized":"2024-09-17T04:34:20.026887+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:16.643583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027557+0000","last_change":"2024-09-17T04:33:59.114304+0000","last_active":"2024-09-17T04:34:20.027557+0000","last_peered":"2024-09-17T04:34:20.027557+0000","last_clean":"2024-09-17T04:34:20.027557+0000","last_became_active":"2024-09-17T04:33:59.113930+0000","last_became_peered":"2024-09-17T04:33:59.113930+0000","last_unstale":"2024-09-17T04:34:20.027557+0000","last_undegraded":"2024-09-17T04:34:20.027557+0000","last_fullsized":"2024-09-17T04:34:20.027557+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:11.433911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"33'8","reported_seq":36,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027173+0000","last_change":"2024-09-17T04:34:00.135067+0000","last_active":"2024-09-17T04:34:20.027173+0000","last_peered":"2024-09-17T04:34:20.027173+0000","last_clean":"2024-09-17T04:34:20.027173+0000","last_became_active":"2024-09-17T04:34:00.134742+0000","last_became_peered":"2024-09-17T04:34:00.134742+0000","last_unstale":"2024-09-17T04:34:20.027173+0000","last_undegraded":"2024-09-17T04:34:20.027173+0000","last_fullsized":"2024-09-17T04:34:20.027173+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:47.268736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.026804+0000","last_change":"2024-09-17T04:34:18.198826+0000","last_active":"2024-09-17T04:34:20.026804+0000","last_peered":"2024-09-17T04:34:20.026804+0000","last_clean":"2024-09-17T04:34:20.026804+0000","last_became_active":"2024-09-17T04:34:18.198509+0000","last_became_peered":"2024-09-17T04:34:18.198509+0000","last_unstale":"2024-09-17T04:34:20.026804+0000","last_undegraded":"2024-09-17T04:34:20.026804+0000","last_fullsized":"2024-09-17T04:34:20.026804+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:34.253604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027967+0000","last_change":"2024-09-17T04:33:59.119426+0000","last_active":"2024-09-17T04:34:20.027967+0000","last_peered":"2024-09-17T04:34:20.027967+0000","last_clean":"2024-09-17T04:34:20.027967+0000","last_became_active":"2024-09-17T04:33:59.118894+0000","last_became_peered":"2024-09-17T04:33:59.118894+0000","last_unstale":"2024-09-17T04:34:20.027967+0000","last_undegraded":"2024-09-17T04:34:20.027967+0000","last_fullsized":"2024-09-17T04:34:20.027967+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:31.864330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"33'5","reported_seq":35,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.026767+0000","last_change":"2024-09-17T04:34:00.144557+0000","last_active":"2024-09-17T04:34:20.026767+0000","last_peered":"2024-09-17T04:34:20.026767+0000","last_clean":"2024-09-17T04:34:20.026767+0000","last_became_active":"2024-09-17T04:34:00.144230+0000","last_became_peered":"2024-09-17T04:34:00.144230+0000","last_unstale":"2024-09-17T04:34:20.026767+0000","last_undegraded":"2024-09-17T04:34:20.026767+0000","last_fullsized":"2024-09-17T04:34:20.026767+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:05:34.986501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184728+0000","last_change":"2024-09-17T04:34:18.200388+0000","last_active":"2024-09-17T04:34:19.184728+0000","last_peered":"2024-09-17T04:34:19.184728+0000","last_clean":"2024-09-17T04:34:19.184728+0000","last_became_active":"2024-09-17T04:34:18.200101+0000","last_became_peered":"2024-09-17T04:34:18.200101+0000","last_unstale":"2024-09-17T04:34:19.184728+0000","last_undegraded":"2024-09-17T04:34:19.184728+0000","last_fullsized":"2024-09-17T04:34:19.184728+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:12.840296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"34'5","reported_seq":36,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027382+0000","last_change":"2024-09-17T04:33:59.117234+0000","last_active":"2024-09-17T04:34:20.027382+0000","last_peered":"2024-09-17T04:34:20.027382+0000","last_clean":"2024-09-17T04:34:20.027382+0000","last_became_active":"2024-09-17T04:33:59.116890+0000","last_became_peered":"2024-09-17T04:33:59.116890+0000","last_unstale":"2024-09-17T04:34:20.027382+0000","last_undegraded":"2024-09-17T04:34:20.027382+0000","last_fullsized":"2024-09-17T04:34:20.027382+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:28.423942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"33'8","reported_seq":36,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.028709+0000","last_change":"2024-09-17T04:34:00.145064+0000","last_active":"2024-09-17T04:34:20.028709+0000","last_peered":"2024-09-17T04:34:20.028709+0000","last_clean":"2024-09-17T04:34:20.028709+0000","last_became_active":"2024-09-17T04:34:00.144786+0000","last_became_peered":"2024-09-17T04:34:00.144786+0000","last_unstale":"2024-09-17T04:34:20.028709+0000","last_undegraded":"2024-09-17T04:34:20.028709+0000","last_fullsized":"2024-09-17T04:34:20.028709+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:15:12.698992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027747+0000","last_change":"2024-09-17T04:34:18.197865+0000","last_active":"2024-09-17T04:34:20.027747+0000","last_peered":"2024-09-17T04:34:20.027747+0000","last_clean":"2024-09-17T04:34:20.027747+0000","last_became_active":"2024-09-17T04:34:18.197546+0000","last_became_peered":"2024-09-17T04:34:18.197546+0000","last_unstale":"2024-09-17T04:34:20.027747+0000","last_undegraded":"2024-09-17T04:34:20.027747+0000","last_fullsized":"2024-09-17T04:34:20.027747+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:42.765620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"34'3","reported_seq":32,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027357+0000","last_change":"2024-09-17T04:33:59.116203+0000","last_active":"2024-09-17T04:34:20.027357+0000","last_peered":"2024-09-17T04:34:20.027357+0000","last_clean":"2024-09-17T04:34:20.027357+0000","last_became_active":"2024-09-17T04:33:59.115805+0000","last_became_peered":"2024-09-17T04:33:59.115805+0000","last_unstale":"2024-09-17T04:34:20.027357+0000","last_undegraded":"2024-09-17T04:34:20.027357+0000","last_fullsized":"2024-09-17T04:34:20.027357+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:23:51.532043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"33'7","reported_seq":35,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.183330+0000","last_change":"2024-09-17T04:34:00.139613+0000","last_active":"2024-09-17T04:34:19.183330+0000","last_peered":"2024-09-17T04:34:19.183330+0000","last_clean":"2024-09-17T04:34:19.183330+0000","last_became_active":"2024-09-17T04:34:00.139292+0000","last_became_peered":"2024-09-17T04:34:00.139292+0000","last_unstale":"2024-09-17T04:34:19.183330+0000","last_undegraded":"2024-09-17T04:34:19.183330+0000","last_fullsized":"2024-09-17T04:34:19.183330+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:34.594760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.028857+0000","last_change":"2024-09-17T04:34:18.203518+0000","last_active":"2024-09-17T04:34:20.028857+0000","last_peered":"2024-09-17T04:34:20.028857+0000","last_clean":"2024-09-17T04:34:20.028857+0000","last_became_active":"2024-09-17T04:34:18.203132+0000","last_became_peered":"2024-09-17T04:34:18.203132+0000","last_unstale":"2024-09-17T04:34:20.028857+0000","last_undegraded":"2024-09-17T04:34:20.028857+0000","last_fullsized":"2024-09-17T04:34:20.028857+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:05.326904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.026758+0000","last_change":"2024-09-17T04:33:59.113577+0000","last_active":"2024-09-17T04:34:20.026758+0000","last_peered":"2024-09-17T04:34:20.026758+0000","last_clean":"2024-09-17T04:34:20.026758+0000","last_became_active":"2024-09-17T04:33:59.113210+0000","last_became_peered":"2024-09-17T04:33:59.113210+0000","last_unstale":"2024-09-17T04:34:20.026758+0000","last_undegraded":"2024-09-17T04:34:20.026758+0000","last_fullsized":"2024-09-17T04:34:20.026758+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:06.764372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"33'2","reported_seq":32,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.026893+0000","last_change":"2024-09-17T04:34:00.134940+0000","last_active":"2024-09-17T04:34:20.026893+0000","last_peered":"2024-09-17T04:34:20.026893+0000","last_clean":"2024-09-17T04:34:20.026893+0000","last_became_active":"2024-09-17T04:34:00.134621+0000","last_became_peered":"2024-09-17T04:34:00.134621+0000","last_unstale":"2024-09-17T04:34:20.026893+0000","last_undegraded":"2024-09-17T04:34:20.026893+0000","last_fullsized":"2024-09-17T04:34:20.026893+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:41.467535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027760+0000","last_change":"2024-09-17T04:34:18.198649+0000","last_active":"2024-09-17T04:34:20.027760+0000","last_peered":"2024-09-17T04:34:20.027760+0000","last_clean":"2024-09-17T04:34:20.027760+0000","last_became_active":"2024-09-17T04:34:18.198317+0000","last_became_peered":"2024-09-17T04:34:18.198317+0000","last_unstale":"2024-09-17T04:34:20.027760+0000","last_undegraded":"2024-09-17T04:34:20.027760+0000","last_fullsized":"2024-09-17T04:34:20.027760+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:35.198530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184350+0000","last_change":"2024-09-17T04:33:59.116189+0000","last_active":"2024-09-17T04:34:19.184350+0000","last_peered":"2024-09-17T04:34:19.184350+0000","last_clean":"2024-09-17T04:34:19.184350+0000","last_became_active":"2024-09-17T04:33:59.115627+0000","last_became_peered":"2024-09-17T04:33:59.115627+0000","last_unstale":"2024-09-17T04:34:19.184350+0000","last_undegraded":"2024-09-17T04:34:19.184350+0000","last_fullsized":"2024-09-17T04:34:19.184350+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:31.619034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"33'9","reported_seq":37,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.183047+0000","last_change":"2024-09-17T04:34:00.133387+0000","last_active":"2024-09-17T04:34:19.183047+0000","last_peered":"2024-09-17T04:34:19.183047+0000","last_clean":"2024-09-17T04:34:19.183047+0000","last_became_active":"2024-09-17T04:34:00.133075+0000","last_became_peered":"2024-09-17T04:34:00.133075+0000","last_unstale":"2024-09-17T04:34:19.183047+0000","last_undegraded":"2024-09-17T04:34:19.183047+0000","last_fullsized":"2024-09-17T04:34:19.183047+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:12.436009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.185308+0000","last_change":"2024-09-17T04:34:18.198846+0000","last_active":"2024-09-17T04:34:19.185308+0000","last_peered":"2024-09-17T04:34:19.185308+0000","last_clean":"2024-09-17T04:34:19.185308+0000","last_became_active":"2024-09-17T04:34:18.198161+0000","last_became_peered":"2024-09-17T04:34:18.198161+0000","last_unstale":"2024-09-17T04:34:19.185308+0000","last_undegraded":"2024-09-17T04:34:19.185308+0000","last_fullsized":"2024-09-17T04:34:19.185308+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:10:27.146711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027292+0000","last_change":"2024-09-17T04:33:59.116483+0000","last_active":"2024-09-17T04:34:20.027292+0000","last_peered":"2024-09-17T04:34:20.027292+0000","last_clean":"2024-09-17T04:34:20.027292+0000","last_became_active":"2024-09-17T04:33:59.116144+0000","last_became_peered":"2024-09-17T04:33:59.116144+0000","last_unstale":"2024-09-17T04:34:20.027292+0000","last_undegraded":"2024-09-17T04:34:20.027292+0000","last_fullsized":"2024-09-17T04:34:20.027292+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:40.142964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"33'8","reported_seq":36,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.183273+0000","last_change":"2024-09-17T04:34:00.134391+0000","last_active":"2024-09-17T04:34:19.183273+0000","last_peered":"2024-09-17T04:34:19.183273+0000","last_clean":"2024-09-17T04:34:19.183273+0000","last_became_active":"2024-09-17T04:34:00.134057+0000","last_became_peered":"2024-09-17T04:34:00.134057+0000","last_unstale":"2024-09-17T04:34:19.183273+0000","last_undegraded":"2024-09-17T04:34:19.183273+0000","last_fullsized":"2024-09-17T04:34:19.183273+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:55.679577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027412+0000","last_change":"2024-09-17T04:34:18.198847+0000","last_active":"2024-09-17T04:34:20.027412+0000","last_peered":"2024-09-17T04:34:20.027412+0000","last_clean":"2024-09-17T04:34:20.027412+0000","last_became_active":"2024-09-17T04:34:18.198040+0000","last_became_peered":"2024-09-17T04:34:18.198040+0000","last_unstale":"2024-09-17T04:34:20.027412+0000","last_undegraded":"2024-09-17T04:34:20.027412+0000","last_fullsized":"2024-09-17T04:34:20.027412+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:22.398337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"34'4","reported_seq":35,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027313+0000","last_change":"2024-09-17T04:33:59.116366+0000","last_active":"2024-09-17T04:34:20.027313+0000","last_peered":"2024-09-17T04:34:20.027313+0000","last_clean":"2024-09-17T04:34:20.027313+0000","last_became_active":"2024-09-17T04:33:59.116060+0000","last_became_peered":"2024-09-17T04:33:59.116060+0000","last_unstale":"2024-09-17T04:34:20.027313+0000","last_undegraded":"2024-09-17T04:34:20.027313+0000","last_fullsized":"2024-09-17T04:34:20.027313+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:23:51.532043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"33'5","reported_seq":33,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.183054+0000","last_change":"2024-09-17T04:34:00.137771+0000","last_active":"2024-09-17T04:34:19.183054+0000","last_peered":"2024-09-17T04:34:19.183054+0000","last_clean":"2024-09-17T04:34:19.183054+0000","last_became_active":"2024-09-17T04:34:00.137424+0000","last_became_peered":"2024-09-17T04:34:00.137424+0000","last_unstale":"2024-09-17T04:34:19.183054+0000","last_undegraded":"2024-09-17T04:34:19.183054+0000","last_fullsized":"2024-09-17T04:34:19.183054+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:59:37.912219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.026754+0000","last_change":"2024-09-17T04:34:18.200348+0000","last_active":"2024-09-17T04:34:20.026754+0000","last_peered":"2024-09-17T04:34:20.026754+0000","last_clean":"2024-09-17T04:34:20.026754+0000","last_became_active":"2024-09-17T04:34:18.200076+0000","last_became_peered":"2024-09-17T04:34:18.200076+0000","last_unstale":"2024-09-17T04:34:20.026754+0000","last_undegraded":"2024-09-17T04:34:20.026754+0000","last_fullsized":"2024-09-17T04:34:20.026754+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:22:31.050516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184895+0000","last_change":"2024-09-17T04:33:59.115490+0000","last_active":"2024-09-17T04:34:19.184895+0000","last_peered":"2024-09-17T04:34:19.184895+0000","last_clean":"2024-09-17T04:34:19.184895+0000","last_became_active":"2024-09-17T04:33:59.115105+0000","last_became_peered":"2024-09-17T04:33:59.115105+0000","last_unstale":"2024-09-17T04:34:19.184895+0000","last_undegraded":"2024-09-17T04:34:19.184895+0000","last_fullsized":"2024-09-17T04:34:19.184895+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:25.941127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"33'7","reported_seq":35,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.183294+0000","last_change":"2024-09-17T04:34:00.131812+0000","last_active":"2024-09-17T04:34:19.183294+0000","last_peered":"2024-09-17T04:34:19.183294+0000","last_clean":"2024-09-17T04:34:19.183294+0000","last_became_active":"2024-09-17T04:34:00.131461+0000","last_became_peered":"2024-09-17T04:34:00.131461+0000","last_unstale":"2024-09-17T04:34:19.183294+0000","last_undegraded":"2024-09-17T04:34:19.183294+0000","last_fullsized":"2024-09-17T04:34:19.183294+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:07.766470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184994+0000","last_change":"2024-09-17T04:34:18.197790+0000","last_active":"2024-09-17T04:34:19.184994+0000","last_peered":"2024-09-17T04:34:19.184994+0000","last_clean":"2024-09-17T04:34:19.184994+0000","last_became_active":"2024-09-17T04:34:18.196003+0000","last_became_peered":"2024-09-17T04:34:18.196003+0000","last_unstale":"2024-09-17T04:34:19.184994+0000","last_undegraded":"2024-09-17T04:34:19.184994+0000","last_fullsized":"2024-09-17T04:34:19.184994+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:22:04.467099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"34'2","reported_seq":32,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184923+0000","last_change":"2024-09-17T04:33:59.112268+0000","last_active":"2024-09-17T04:34:19.184923+0000","last_peered":"2024-09-17T04:34:19.184923+0000","last_clean":"2024-09-17T04:34:19.184923+0000","last_became_active":"2024-09-17T04:33:59.111822+0000","last_became_peered":"2024-09-17T04:33:59.111822+0000","last_unstale":"2024-09-17T04:34:19.184923+0000","last_undegraded":"2024-09-17T04:34:19.184923+0000","last_fullsized":"2024-09-17T04:34:19.184923+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:21:44.164301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"33'6","reported_seq":34,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027843+0000","last_change":"2024-09-17T04:34:00.138877+0000","last_active":"2024-09-17T04:34:20.027843+0000","last_peered":"2024-09-17T04:34:20.027843+0000","last_clean":"2024-09-17T04:34:20.027843+0000","last_became_active":"2024-09-17T04:34:00.138529+0000","last_became_peered":"2024-09-17T04:34:00.138529+0000","last_unstale":"2024-09-17T04:34:20.027843+0000","last_undegraded":"2024-09-17T04:34:20.027843+0000","last_fullsized":"2024-09-17T04:34:20.027843+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:43:32.412802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.183622+0000","last_change":"2024-09-17T04:34:18.188430+0000","last_active":"2024-09-17T04:34:19.183622+0000","last_peered":"2024-09-17T04:34:19.183622+0000","last_clean":"2024-09-17T04:34:19.183622+0000","last_became_active":"2024-09-17T04:34:18.188114+0000","last_became_peered":"2024-09-17T04:34:18.188114+0000","last_unstale":"2024-09-17T04:34:19.183622+0000","last_undegraded":"2024-09-17T04:34:19.183622+0000","last_fullsized":"2024-09-17T04:34:19.183622+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:11:06.116729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"34'3","reported_seq":33,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.185316+0000","last_change":"2024-09-17T04:33:59.109283+0000","last_active":"2024-09-17T04:34:19.185316+0000","last_peered":"2024-09-17T04:34:19.185316+0000","last_clean":"2024-09-17T04:34:19.185316+0000","last_became_active":"2024-09-17T04:33:59.108953+0000","last_became_peered":"2024-09-17T04:33:59.108953+0000","last_unstale":"2024-09-17T04:34:19.185316+0000","last_undegraded":"2024-09-17T04:34:19.185316+0000","last_fullsized":"2024-09-17T04:34:19.185316+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:39.786291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"33'9","reported_seq":37,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.028609+0000","last_change":"2024-09-17T04:34:00.143589+0000","last_active":"2024-09-17T04:34:20.028609+0000","last_peered":"2024-09-17T04:34:20.028609+0000","last_clean":"2024-09-17T04:34:20.028609+0000","last_became_active":"2024-09-17T04:34:00.143288+0000","last_became_peered":"2024-09-17T04:34:00.143288+0000","last_unstale":"2024-09-17T04:34:20.028609+0000","last_undegraded":"2024-09-17T04:34:20.028609+0000","last_fullsized":"2024-09-17T04:34:20.028609+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:20.470320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184653+0000","last_change":"2024-09-17T04:34:18.186465+0000","last_active":"2024-09-17T04:34:19.184653+0000","last_peered":"2024-09-17T04:34:19.184653+0000","last_clean":"2024-09-17T04:34:19.184653+0000","last_became_active":"2024-09-17T04:34:18.186153+0000","last_became_peered":"2024-09-17T04:34:18.186153+0000","last_unstale":"2024-09-17T04:34:19.184653+0000","last_undegraded":"2024-09-17T04:34:19.184653+0000","last_fullsized":"2024-09-17T04:34:19.184653+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:07.123655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"34'2","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027318+0000","last_change":"2024-09-17T04:33:59.118082+0000","last_active":"2024-09-17T04:34:20.027318+0000","last_peered":"2024-09-17T04:34:20.027318+0000","last_clean":"2024-09-17T04:34:20.027318+0000","last_became_active":"2024-09-17T04:33:59.117749+0000","last_became_peered":"2024-09-17T04:33:59.117749+0000","last_unstale":"2024-09-17T04:34:20.027318+0000","last_undegraded":"2024-09-17T04:34:20.027318+0000","last_fullsized":"2024-09-17T04:34:20.027318+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:22:09.267634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"33'5","reported_seq":34,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.185176+0000","last_change":"2024-09-17T04:34:00.140363+0000","last_active":"2024-09-17T04:34:19.185176+0000","last_peered":"2024-09-17T04:34:19.185176+0000","last_clean":"2024-09-17T04:34:19.185176+0000","last_became_active":"2024-09-17T04:34:00.138809+0000","last_became_peered":"2024-09-17T04:34:00.138809+0000","last_unstale":"2024-09-17T04:34:19.185176+0000","last_undegraded":"2024-09-17T04:34:19.185176+0000","last_fullsized":"2024-09-17T04:34:19.185176+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:32.041659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027947+0000","last_change":"2024-09-17T04:34:18.196669+0000","last_active":"2024-09-17T04:34:20.027947+0000","last_peered":"2024-09-17T04:34:20.027947+0000","last_clean":"2024-09-17T04:34:20.027947+0000","last_became_active":"2024-09-17T04:34:18.196298+0000","last_became_peered":"2024-09-17T04:34:18.196298+0000","last_unstale":"2024-09-17T04:34:20.027947+0000","last_undegraded":"2024-09-17T04:34:20.027947+0000","last_fullsized":"2024-09-17T04:34:20.027947+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:05:29.822021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"34'15","reported_seq":43,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184305+0000","last_change":"2024-09-17T04:33:59.122217+0000","last_active":"2024-09-17T04:34:19.184305+0000","last_peered":"2024-09-17T04:34:19.184305+0000","last_clean":"2024-09-17T04:34:19.184305+0000","last_became_active":"2024-09-17T04:33:59.121866+0000","last_became_peered":"2024-09-17T04:33:59.121866+0000","last_unstale":"2024-09-17T04:34:19.184305+0000","last_undegraded":"2024-09-17T04:34:19.184305+0000","last_fullsized":"2024-09-17T04:34:19.184305+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:33.592661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"33'6","reported_seq":34,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027282+0000","last_change":"2024-09-17T04:34:00.147713+0000","last_active":"2024-09-17T04:34:20.027282+0000","last_peered":"2024-09-17T04:34:20.027282+0000","last_clean":"2024-09-17T04:34:20.027282+0000","last_became_active":"2024-09-17T04:34:00.147099+0000","last_became_peered":"2024-09-17T04:34:00.147099+0000","last_unstale":"2024-09-17T04:34:20.027282+0000","last_undegraded":"2024-09-17T04:34:20.027282+0000","last_fullsized":"2024-09-17T04:34:20.027282+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:32.252271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.026890+0000","last_change":"2024-09-17T04:34:18.197975+0000","last_active":"2024-09-17T04:34:20.026890+0000","last_peered":"2024-09-17T04:34:20.026890+0000","last_clean":"2024-09-17T04:34:20.026890+0000","last_became_active":"2024-09-17T04:34:18.197392+0000","last_became_peered":"2024-09-17T04:34:18.197392+0000","last_unstale":"2024-09-17T04:34:20.026890+0000","last_undegraded":"2024-09-17T04:34:20.026890+0000","last_fullsized":"2024-09-17T04:34:20.026890+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:24:40.242567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184491+0000","last_change":"2024-09-17T04:33:59.120796+0000","last_active":"2024-09-17T04:34:19.184491+0000","last_peered":"2024-09-17T04:34:19.184491+0000","last_clean":"2024-09-17T04:34:19.184491+0000","last_became_active":"2024-09-17T04:33:59.120478+0000","last_became_peered":"2024-09-17T04:33:59.120478+0000","last_unstale":"2024-09-17T04:34:19.184491+0000","last_undegraded":"2024-09-17T04:34:19.184491+0000","last_fullsized":"2024-09-17T04:34:19.184491+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:54.677479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"33'6","reported_seq":36,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027110+0000","last_change":"2024-09-17T04:34:00.148329+0000","last_active":"2024-09-17T04:34:20.027110+0000","last_peered":"2024-09-17T04:34:20.027110+0000","last_clean":"2024-09-17T04:34:20.027110+0000","last_became_active":"2024-09-17T04:34:00.147988+0000","last_became_peered":"2024-09-17T04:34:00.147988+0000","last_unstale":"2024-09-17T04:34:20.027110+0000","last_undegraded":"2024-09-17T04:34:20.027110+0000","last_fullsized":"2024-09-17T04:34:20.027110+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:57:32.869825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.185050+0000","last_change":"2024-09-17T04:34:18.191128+0000","last_active":"2024-09-17T04:34:19.185050+0000","last_peered":"2024-09-17T04:34:19.185050+0000","last_clean":"2024-09-17T04:34:19.185050+0000","last_became_active":"2024-09-17T04:34:18.190784+0000","last_became_peered":"2024-09-17T04:34:18.190784+0000","last_unstale":"2024-09-17T04:34:19.185050+0000","last_undegraded":"2024-09-17T04:34:19.185050+0000","last_fullsized":"2024-09-17T04:34:19.185050+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:37:16.551154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027135+0000","last_change":"2024-09-17T04:33:59.116224+0000","last_active":"2024-09-17T04:34:20.027135+0000","last_peered":"2024-09-17T04:34:20.027135+0000","last_clean":"2024-09-17T04:34:20.027135+0000","last_became_active":"2024-09-17T04:33:59.115862+0000","last_became_peered":"2024-09-17T04:33:59.115862+0000","last_unstale":"2024-09-17T04:34:20.027135+0000","last_undegraded":"2024-09-17T04:34:20.027135+0000","last_fullsized":"2024-09-17T04:34:20.027135+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:22:13.092980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"33'15","reported_seq":42,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.185376+0000","last_change":"2024-09-17T04:34:00.145327+0000","last_active":"2024-09-17T04:34:19.185376+0000","last_peered":"2024-09-17T04:34:19.185376+0000","last_clean":"2024-09-17T04:34:19.185376+0000","last_became_active":"2024-09-17T04:34:00.142809+0000","last_became_peered":"2024-09-17T04:34:00.142809+0000","last_unstale":"2024-09-17T04:34:19.185376+0000","last_undegraded":"2024-09-17T04:34:19.185376+0000","last_fullsized":"2024-09-17T04:34:19.185376+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:57.918228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.026794+0000","last_change":"2024-09-17T04:34:18.196770+0000","last_active":"2024-09-17T04:34:20.026794+0000","last_peered":"2024-09-17T04:34:20.026794+0000","last_clean":"2024-09-17T04:34:20.026794+0000","last_became_active":"2024-09-17T04:34:18.196428+0000","last_became_peered":"2024-09-17T04:34:18.196428+0000","last_unstale":"2024-09-17T04:34:20.026794+0000","last_undegraded":"2024-09-17T04:34:20.026794+0000","last_fullsized":"2024-09-17T04:34:20.026794+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:04.801206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"34'5","reported_seq":35,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.185026+0000","last_change":"2024-09-17T04:33:59.109143+0000","last_active":"2024-09-17T04:34:19.185026+0000","last_peered":"2024-09-17T04:34:19.185026+0000","last_clean":"2024-09-17T04:34:19.185026+0000","last_became_active":"2024-09-17T04:33:59.108847+0000","last_became_peered":"2024-09-17T04:33:59.108847+0000","last_unstale":"2024-09-17T04:34:19.185026+0000","last_undegraded":"2024-09-17T04:34:19.185026+0000","last_fullsized":"2024-09-17T04:34:19.185026+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:18.019734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"33'10","reported_seq":38,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.026985+0000","last_change":"2024-09-17T04:34:00.132724+0000","last_active":"2024-09-17T04:34:20.026985+0000","last_peered":"2024-09-17T04:34:20.026985+0000","last_clean":"2024-09-17T04:34:20.026985+0000","last_became_active":"2024-09-17T04:34:00.131882+0000","last_became_peered":"2024-09-17T04:34:00.131882+0000","last_unstale":"2024-09-17T04:34:20.026985+0000","last_undegraded":"2024-09-17T04:34:20.026985+0000","last_fullsized":"2024-09-17T04:34:20.026985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:38.015410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184493+0000","last_change":"2024-09-17T04:34:18.188267+0000","last_active":"2024-09-17T04:34:19.184493+0000","last_peered":"2024-09-17T04:34:19.184493+0000","last_clean":"2024-09-17T04:34:19.184493+0000","last_became_active":"2024-09-17T04:34:18.187965+0000","last_became_peered":"2024-09-17T04:34:18.187965+0000","last_unstale":"2024-09-17T04:34:19.184493+0000","last_undegraded":"2024-09-17T04:34:19.184493+0000","last_fullsized":"2024-09-17T04:34:19.184493+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:33.578703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"34'23","reported_seq":51,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.185126+0000","last_change":"2024-09-17T04:33:59.124191+0000","last_active":"2024-09-17T04:34:19.185126+0000","last_peered":"2024-09-17T04:34:19.185126+0000","last_clean":"2024-09-17T04:34:19.185126+0000","last_became_active":"2024-09-17T04:33:59.123823+0000","last_became_peered":"2024-09-17T04:33:59.123823+0000","last_unstale":"2024-09-17T04:34:19.185126+0000","last_undegraded":"2024-09-17T04:34:19.185126+0000","last_fullsized":"2024-09-17T04:34:19.185126+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:39:49.197130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"33'6","reported_seq":34,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.028036+0000","last_change":"2024-09-17T04:34:00.146174+0000","last_active":"2024-09-17T04:34:20.028036+0000","last_peered":"2024-09-17T04:34:20.028036+0000","last_clean":"2024-09-17T04:34:20.028036+0000","last_became_active":"2024-09-17T04:34:00.145752+0000","last_became_peered":"2024-09-17T04:34:00.145752+0000","last_unstale":"2024-09-17T04:34:20.028036+0000","last_undegraded":"2024-09-17T04:34:20.028036+0000","last_fullsized":"2024-09-17T04:34:20.028036+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:20:51.439787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027189+0000","last_change":"2024-09-17T04:34:18.198218+0000","last_active":"2024-09-17T04:34:20.027189+0000","last_peered":"2024-09-17T04:34:20.027189+0000","last_clean":"2024-09-17T04:34:20.027189+0000","last_became_active":"2024-09-17T04:34:18.197860+0000","last_became_peered":"2024-09-17T04:34:18.197860+0000","last_unstale":"2024-09-17T04:34:20.027189+0000","last_undegraded":"2024-09-17T04:34:20.027189+0000","last_fullsized":"2024-09-17T04:34:20.027189+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:43:50.521474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"34'5","reported_seq":34,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.028240+0000","last_change":"2024-09-17T04:33:59.112913+0000","last_active":"2024-09-17T04:34:20.028240+0000","last_peered":"2024-09-17T04:34:20.028240+0000","last_clean":"2024-09-17T04:34:20.028240+0000","last_became_active":"2024-09-17T04:33:59.112540+0000","last_became_peered":"2024-09-17T04:33:59.112540+0000","last_unstale":"2024-09-17T04:34:20.028240+0000","last_undegraded":"2024-09-17T04:34:20.028240+0000","last_fullsized":"2024-09-17T04:34:20.028240+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:33.592661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"33'11","reported_seq":38,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184914+0000","last_change":"2024-09-17T04:34:00.148749+0000","last_active":"2024-09-17T04:34:19.184914+0000","last_peered":"2024-09-17T04:34:19.184914+0000","last_clean":"2024-09-17T04:34:19.184914+0000","last_became_active":"2024-09-17T04:34:00.147951+0000","last_became_peered":"2024-09-17T04:34:00.147951+0000","last_unstale":"2024-09-17T04:34:19.184914+0000","last_undegraded":"2024-09-17T04:34:19.184914+0000","last_fullsized":"2024-09-17T04:34:19.184914+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:20:51.439787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.185083+0000","last_change":"2024-09-17T04:34:18.198654+0000","last_active":"2024-09-17T04:34:19.185083+0000","last_peered":"2024-09-17T04:34:19.185083+0000","last_clean":"2024-09-17T04:34:19.185083+0000","last_became_active":"2024-09-17T04:34:18.196077+0000","last_became_peered":"2024-09-17T04:34:18.196077+0000","last_unstale":"2024-09-17T04:34:19.185083+0000","last_undegraded":"2024-09-17T04:34:19.185083+0000","last_fullsized":"2024-09-17T04:34:19.185083+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:38.401410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027293+0000","last_change":"2024-09-17T04:33:59.112992+0000","last_active":"2024-09-17T04:34:20.027293+0000","last_peered":"2024-09-17T04:34:20.027293+0000","last_clean":"2024-09-17T04:34:20.027293+0000","last_became_active":"2024-09-17T04:33:59.112657+0000","last_became_peered":"2024-09-17T04:33:59.112657+0000","last_unstale":"2024-09-17T04:34:20.027293+0000","last_undegraded":"2024-09-17T04:34:20.027293+0000","last_fullsized":"2024-09-17T04:34:20.027293+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:06.203780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"33'9","reported_seq":36,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184967+0000","last_change":"2024-09-17T04:34:00.150547+0000","last_active":"2024-09-17T04:34:19.184967+0000","last_peered":"2024-09-17T04:34:19.184967+0000","last_clean":"2024-09-17T04:34:19.184967+0000","last_became_active":"2024-09-17T04:34:00.146052+0000","last_became_peered":"2024-09-17T04:34:00.146052+0000","last_unstale":"2024-09-17T04:34:19.184967+0000","last_undegraded":"2024-09-17T04:34:19.184967+0000","last_fullsized":"2024-09-17T04:34:19.184967+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:18.597551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.183337+0000","last_change":"2024-09-17T04:34:18.197573+0000","last_active":"2024-09-17T04:34:19.183337+0000","last_peered":"2024-09-17T04:34:19.183337+0000","last_clean":"2024-09-17T04:34:19.183337+0000","last_became_active":"2024-09-17T04:34:18.197228+0000","last_became_peered":"2024-09-17T04:34:18.197228+0000","last_unstale":"2024-09-17T04:34:19.183337+0000","last_undegraded":"2024-09-17T04:34:19.183337+0000","last_fullsized":"2024-09-17T04:34:19.183337+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:36.002032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.185096+0000","last_change":"2024-09-17T04:33:59.120720+0000","last_active":"2024-09-17T04:34:19.185096+0000","last_peered":"2024-09-17T04:34:19.185096+0000","last_clean":"2024-09-17T04:34:19.185096+0000","last_became_active":"2024-09-17T04:33:59.120386+0000","last_became_peered":"2024-09-17T04:33:59.120386+0000","last_unstale":"2024-09-17T04:34:19.185096+0000","last_undegraded":"2024-09-17T04:34:19.185096+0000","last_fullsized":"2024-09-17T04:34:19.185096+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:43.682967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"33'10","reported_seq":38,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027187+0000","last_change":"2024-09-17T04:34:00.149518+0000","last_active":"2024-09-17T04:34:20.027187+0000","last_peered":"2024-09-17T04:34:20.027187+0000","last_clean":"2024-09-17T04:34:20.027187+0000","last_became_active":"2024-09-17T04:34:00.149133+0000","last_became_peered":"2024-09-17T04:34:00.149133+0000","last_unstale":"2024-09-17T04:34:20.027187+0000","last_undegraded":"2024-09-17T04:34:20.027187+0000","last_fullsized":"2024-09-17T04:34:20.027187+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:01.405073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.028879+0000","last_change":"2024-09-17T04:34:18.203928+0000","last_active":"2024-09-17T04:34:20.028879+0000","last_peered":"2024-09-17T04:34:20.028879+0000","last_clean":"2024-09-17T04:34:20.028879+0000","last_became_active":"2024-09-17T04:34:18.203161+0000","last_became_peered":"2024-09-17T04:34:18.203161+0000","last_unstale":"2024-09-17T04:34:20.028879+0000","last_undegraded":"2024-09-17T04:34:20.028879+0000","last_fullsized":"2024-09-17T04:34:20.028879+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:10:32.440074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184307+0000","last_change":"2024-09-17T04:33:59.109225+0000","last_active":"2024-09-17T04:34:19.184307+0000","last_peered":"2024-09-17T04:34:19.184307+0000","last_clean":"2024-09-17T04:34:19.184307+0000","last_became_active":"2024-09-17T04:33:59.108890+0000","last_became_peered":"2024-09-17T04:33:59.108890+0000","last_unstale":"2024-09-17T04:34:19.184307+0000","last_undegraded":"2024-09-17T04:34:19.184307+0000","last_fullsized":"2024-09-17T04:34:19.184307+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:59.447364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"33'10","reported_seq":38,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.026621+0000","last_change":"2024-09-17T04:34:00.137017+0000","last_active":"2024-09-17T04:34:20.026621+0000","last_peered":"2024-09-17T04:34:20.026621+0000","last_clean":"2024-09-17T04:34:20.026621+0000","last_became_active":"2024-09-17T04:34:00.136469+0000","last_became_peered":"2024-09-17T04:34:00.136469+0000","last_unstale":"2024-09-17T04:34:20.026621+0000","last_undegraded":"2024-09-17T04:34:20.026621+0000","last_fullsized":"2024-09-17T04:34:20.026621+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:09:46.692534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184710+0000","last_change":"2024-09-17T04:34:18.198672+0000","last_active":"2024-09-17T04:34:19.184710+0000","last_peered":"2024-09-17T04:34:19.184710+0000","last_clean":"2024-09-17T04:34:19.184710+0000","last_became_active":"2024-09-17T04:34:18.198229+0000","last_became_peered":"2024-09-17T04:34:18.198229+0000","last_unstale":"2024-09-17T04:34:19.184710+0000","last_undegraded":"2024-09-17T04:34:19.184710+0000","last_fullsized":"2024-09-17T04:34:19.184710+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:06:43.317906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184355+0000","last_change":"2024-09-17T04:33:59.121968+0000","last_active":"2024-09-17T04:34:19.184355+0000","last_peered":"2024-09-17T04:34:19.184355+0000","last_clean":"2024-09-17T04:34:19.184355+0000","last_became_active":"2024-09-17T04:33:59.121152+0000","last_became_peered":"2024-09-17T04:33:59.121152+0000","last_unstale":"2024-09-17T04:34:19.184355+0000","last_undegraded":"2024-09-17T04:34:19.184355+0000","last_fullsized":"2024-09-17T04:34:19.184355+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:02:14.225662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"33'8","reported_seq":37,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184298+0000","last_change":"2024-09-17T04:34:00.132375+0000","last_active":"2024-09-17T04:34:19.184298+0000","last_peered":"2024-09-17T04:34:19.184298+0000","last_clean":"2024-09-17T04:34:19.184298+0000","last_became_active":"2024-09-17T04:34:00.131490+0000","last_became_peered":"2024-09-17T04:34:00.131490+0000","last_unstale":"2024-09-17T04:34:19.184298+0000","last_undegraded":"2024-09-17T04:34:19.184298+0000","last_fullsized":"2024-09-17T04:34:19.184298+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:34:15.449975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.185181+0000","last_change":"2024-09-17T04:34:18.198909+0000","last_active":"2024-09-17T04:34:19.185181+0000","last_peered":"2024-09-17T04:34:19.185181+0000","last_clean":"2024-09-17T04:34:19.185181+0000","last_became_active":"2024-09-17T04:34:18.198547+0000","last_became_peered":"2024-09-17T04:34:18.198547+0000","last_unstale":"2024-09-17T04:34:19.185181+0000","last_undegraded":"2024-09-17T04:34:19.185181+0000","last_fullsized":"2024-09-17T04:34:19.185181+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:44:05.269562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.185408+0000","last_change":"2024-09-17T04:33:59.117052+0000","last_active":"2024-09-17T04:34:19.185408+0000","last_peered":"2024-09-17T04:34:19.185408+0000","last_clean":"2024-09-17T04:34:19.185408+0000","last_became_active":"2024-09-17T04:33:59.116738+0000","last_became_peered":"2024-09-17T04:33:59.116738+0000","last_unstale":"2024-09-17T04:34:19.185408+0000","last_undegraded":"2024-09-17T04:34:19.185408+0000","last_fullsized":"2024-09-17T04:34:19.185408+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:09.152247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"33'8","reported_seq":36,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027813+0000","last_change":"2024-09-17T04:34:00.142714+0000","last_active":"2024-09-17T04:34:20.027813+0000","last_peered":"2024-09-17T04:34:20.027813+0000","last_clean":"2024-09-17T04:34:20.027813+0000","last_became_active":"2024-09-17T04:34:00.142287+0000","last_became_peered":"2024-09-17T04:34:00.142287+0000","last_unstale":"2024-09-17T04:34:20.027813+0000","last_undegraded":"2024-09-17T04:34:20.027813+0000","last_fullsized":"2024-09-17T04:34:20.027813+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:51.136426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.026573+0000","last_change":"2024-09-17T04:34:18.190889+0000","last_active":"2024-09-17T04:34:20.026573+0000","last_peered":"2024-09-17T04:34:20.026573+0000","last_clean":"2024-09-17T04:34:20.026573+0000","last_became_active":"2024-09-17T04:34:18.190549+0000","last_became_peered":"2024-09-17T04:34:18.190549+0000","last_unstale":"2024-09-17T04:34:20.026573+0000","last_undegraded":"2024-09-17T04:34:20.026573+0000","last_fullsized":"2024-09-17T04:34:20.026573+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:44:01.865305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184679+0000","last_change":"2024-09-17T04:33:59.114365+0000","last_active":"2024-09-17T04:34:19.184679+0000","last_peered":"2024-09-17T04:34:19.184679+0000","last_clean":"2024-09-17T04:34:19.184679+0000","last_became_active":"2024-09-17T04:33:59.114058+0000","last_became_peered":"2024-09-17T04:33:59.114058+0000","last_unstale":"2024-09-17T04:34:19.184679+0000","last_undegraded":"2024-09-17T04:34:19.184679+0000","last_fullsized":"2024-09-17T04:34:19.184679+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:43:41.290086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"33'7","reported_seq":37,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.028288+0000","last_change":"2024-09-17T04:34:00.143488+0000","last_active":"2024-09-17T04:34:20.028288+0000","last_peered":"2024-09-17T04:34:20.028288+0000","last_clean":"2024-09-17T04:34:20.028288+0000","last_became_active":"2024-09-17T04:34:00.143166+0000","last_became_peered":"2024-09-17T04:34:00.143166+0000","last_unstale":"2024-09-17T04:34:20.028288+0000","last_undegraded":"2024-09-17T04:34:20.028288+0000","last_fullsized":"2024-09-17T04:34:20.028288+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:16.144931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027424+0000","last_change":"2024-09-17T04:34:18.196775+0000","last_active":"2024-09-17T04:34:20.027424+0000","last_peered":"2024-09-17T04:34:20.027424+0000","last_clean":"2024-09-17T04:34:20.027424+0000","last_became_active":"2024-09-17T04:34:18.196459+0000","last_became_peered":"2024-09-17T04:34:18.196459+0000","last_unstale":"2024-09-17T04:34:20.027424+0000","last_undegraded":"2024-09-17T04:34:20.027424+0000","last_fullsized":"2024-09-17T04:34:20.027424+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:16.026901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184915+0000","last_change":"2024-09-17T04:34:18.199048+0000","last_active":"2024-09-17T04:34:19.184915+0000","last_peered":"2024-09-17T04:34:19.184915+0000","last_clean":"2024-09-17T04:34:19.184915+0000","last_became_active":"2024-09-17T04:34:18.198718+0000","last_became_peered":"2024-09-17T04:34:18.198718+0000","last_unstale":"2024-09-17T04:34:19.184915+0000","last_undegraded":"2024-09-17T04:34:19.184915+0000","last_fullsized":"2024-09-17T04:34:19.184915+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:51.050637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'3","reported_seq":32,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.183413+0000","last_change":"2024-09-17T04:33:59.119452+0000","last_active":"2024-09-17T04:34:19.183413+0000","last_peered":"2024-09-17T04:34:19.183413+0000","last_clean":"2024-09-17T04:34:19.183413+0000","last_became_active":"2024-09-17T04:33:59.119108+0000","last_became_peered":"2024-09-17T04:33:59.119108+0000","last_unstale":"2024-09-17T04:34:19.183413+0000","last_undegraded":"2024-09-17T04:34:19.183413+0000","last_fullsized":"2024-09-17T04:34:19.183413+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:52.325898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"33'15","reported_seq":43,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.028152+0000","last_change":"2024-09-17T04:34:00.136203+0000","last_active":"2024-09-17T04:34:20.028152+0000","last_peered":"2024-09-17T04:34:20.028152+0000","last_clean":"2024-09-17T04:34:20.028152+0000","last_became_active":"2024-09-17T04:34:00.135871+0000","last_became_peered":"2024-09-17T04:34:00.135871+0000","last_unstale":"2024-09-17T04:34:20.028152+0000","last_undegraded":"2024-09-17T04:34:20.028152+0000","last_fullsized":"2024-09-17T04:34:20.028152+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:00.886399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.185037+0000","last_change":"2024-09-17T04:34:18.192477+0000","last_active":"2024-09-17T04:34:19.185037+0000","last_peered":"2024-09-17T04:34:19.185037+0000","last_clean":"2024-09-17T04:34:19.185037+0000","last_became_active":"2024-09-17T04:34:18.192215+0000","last_became_peered":"2024-09-17T04:34:18.192215+0000","last_unstale":"2024-09-17T04:34:19.185037+0000","last_undegraded":"2024-09-17T04:34:19.185037+0000","last_fullsized":"2024-09-17T04:34:19.185037+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:30.185130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"34'3","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184548+0000","last_change":"2024-09-17T04:33:59.124080+0000","last_active":"2024-09-17T04:34:19.184548+0000","last_peered":"2024-09-17T04:34:19.184548+0000","last_clean":"2024-09-17T04:34:19.184548+0000","last_became_active":"2024-09-17T04:33:59.123624+0000","last_became_peered":"2024-09-17T04:33:59.123624+0000","last_unstale":"2024-09-17T04:34:19.184548+0000","last_undegraded":"2024-09-17T04:34:19.184548+0000","last_fullsized":"2024-09-17T04:34:19.184548+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:52.054448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"33'9","reported_seq":38,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.185005+0000","last_change":"2024-09-17T04:34:00.148658+0000","last_active":"2024-09-17T04:34:19.185005+0000","last_peered":"2024-09-17T04:34:19.185005+0000","last_clean":"2024-09-17T04:34:19.185005+0000","last_became_active":"2024-09-17T04:34:00.148350+0000","last_became_peered":"2024-09-17T04:34:00.148350+0000","last_unstale":"2024-09-17T04:34:19.185005+0000","last_undegraded":"2024-09-17T04:34:19.185005+0000","last_fullsized":"2024-09-17T04:34:19.185005+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:58:49.566704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":32,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027227+0000","last_change":"2024-09-17T04:33:59.113364+0000","last_active":"2024-09-17T04:34:20.027227+0000","last_peered":"2024-09-17T04:34:20.027227+0000","last_clean":"2024-09-17T04:34:20.027227+0000","last_became_active":"2024-09-17T04:33:59.112021+0000","last_became_peered":"2024-09-17T04:33:59.112021+0000","last_unstale":"2024-09-17T04:34:20.027227+0000","last_undegraded":"2024-09-17T04:34:20.027227+0000","last_fullsized":"2024-09-17T04:34:20.027227+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:53:00.824171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"33'6","reported_seq":34,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027537+0000","last_change":"2024-09-17T04:34:00.143656+0000","last_active":"2024-09-17T04:34:20.027537+0000","last_peered":"2024-09-17T04:34:20.027537+0000","last_clean":"2024-09-17T04:34:20.027537+0000","last_became_active":"2024-09-17T04:34:00.143327+0000","last_became_peered":"2024-09-17T04:34:00.143327+0000","last_unstale":"2024-09-17T04:34:20.027537+0000","last_undegraded":"2024-09-17T04:34:20.027537+0000","last_fullsized":"2024-09-17T04:34:20.027537+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:41.010278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.028341+0000","last_change":"2024-09-17T04:34:18.197014+0000","last_active":"2024-09-17T04:34:20.028341+0000","last_peered":"2024-09-17T04:34:20.028341+0000","last_clean":"2024-09-17T04:34:20.028341+0000","last_became_active":"2024-09-17T04:34:18.196679+0000","last_became_peered":"2024-09-17T04:34:18.196679+0000","last_unstale":"2024-09-17T04:34:20.028341+0000","last_undegraded":"2024-09-17T04:34:20.028341+0000","last_fullsized":"2024-09-17T04:34:20.028341+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:06:57.487864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1f","version":"34'8","reported_seq":37,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027658+0000","last_change":"2024-09-17T04:33:59.113533+0000","last_active":"2024-09-17T04:34:20.027658+0000","last_peered":"2024-09-17T04:34:20.027658+0000","last_clean":"2024-09-17T04:34:20.027658+0000","last_became_active":"2024-09-17T04:33:59.113062+0000","last_became_peered":"2024-09-17T04:33:59.113062+0000","last_unstale":"2024-09-17T04:34:20.027658+0000","last_undegraded":"2024-09-17T04:34:20.027658+0000","last_fullsized":"2024-09-17T04:34:20.027658+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:58.089921+0000","last_clean_scrub_stamp":"2024-09-17T04:33:58.089921+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:00.673571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"33'5","reported_seq":35,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027062+0000","last_change":"2024-09-17T04:34:00.139991+0000","last_active":"2024-09-17T04:34:20.027062+0000","last_peered":"2024-09-17T04:34:20.027062+0000","last_clean":"2024-09-17T04:34:20.027062+0000","last_became_active":"2024-09-17T04:34:00.139640+0000","last_became_peered":"2024-09-17T04:34:00.139640+0000","last_unstale":"2024-09-17T04:34:20.027062+0000","last_undegraded":"2024-09-17T04:34:20.027062+0000","last_fullsized":"2024-09-17T04:34:20.027062+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:41.010278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027860+0000","last_change":"2024-09-17T04:34:18.198930+0000","last_active":"2024-09-17T04:34:20.027860+0000","last_peered":"2024-09-17T04:34:20.027860+0000","last_clean":"2024-09-17T04:34:20.027860+0000","last_became_active":"2024-09-17T04:34:18.198620+0000","last_became_peered":"2024-09-17T04:34:18.198620+0000","last_unstale":"2024-09-17T04:34:20.027860+0000","last_undegraded":"2024-09-17T04:34:20.027860+0000","last_fullsized":"2024-09-17T04:34:20.027860+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:34:17.200691+0000","last_clean_scrub_stamp":"2024-09-17T04:34:17.200691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:12.426856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"33'9","reported_seq":39,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.026982+0000","last_change":"2024-09-17T04:34:00.148182+0000","last_active":"2024-09-17T04:34:20.026982+0000","last_peered":"2024-09-17T04:34:20.026982+0000","last_clean":"2024-09-17T04:34:20.026982+0000","last_became_active":"2024-09-17T04:34:00.147537+0000","last_became_peered":"2024-09-17T04:34:00.147537+0000","last_unstale":"2024-09-17T04:34:20.026982+0000","last_undegraded":"2024-09-17T04:34:20.026982+0000","last_fullsized":"2024-09-17T04:34:20.026982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:54.318184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"33'12","reported_seq":40,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027068+0000","last_change":"2024-09-17T04:34:00.140013+0000","last_active":"2024-09-17T04:34:20.027068+0000","last_peered":"2024-09-17T04:34:20.027068+0000","last_clean":"2024-09-17T04:34:20.027068+0000","last_became_active":"2024-09-17T04:34:00.139703+0000","last_became_peered":"2024-09-17T04:34:00.139703+0000","last_unstale":"2024-09-17T04:34:20.027068+0000","last_undegraded":"2024-09-17T04:34:20.027068+0000","last_fullsized":"2024-09-17T04:34:20.027068+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:39:50.199228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.22","version":"33'8","reported_seq":38,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.028132+0000","last_change":"2024-09-17T04:34:00.131809+0000","last_active":"2024-09-17T04:34:20.028132+0000","last_peered":"2024-09-17T04:34:20.028132+0000","last_clean":"2024-09-17T04:34:20.028132+0000","last_became_active":"2024-09-17T04:34:00.131532+0000","last_became_peered":"2024-09-17T04:34:00.131532+0000","last_unstale":"2024-09-17T04:34:20.028132+0000","last_undegraded":"2024-09-17T04:34:20.028132+0000","last_fullsized":"2024-09-17T04:34:20.028132+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:54.247439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.23","version":"33'7","reported_seq":36,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184466+0000","last_change":"2024-09-17T04:34:00.146595+0000","last_active":"2024-09-17T04:34:19.184466+0000","last_peered":"2024-09-17T04:34:19.184466+0000","last_clean":"2024-09-17T04:34:19.184466+0000","last_became_active":"2024-09-17T04:34:00.145812+0000","last_became_peered":"2024-09-17T04:34:00.145812+0000","last_unstale":"2024-09-17T04:34:19.184466+0000","last_undegraded":"2024-09-17T04:34:19.184466+0000","last_fullsized":"2024-09-17T04:34:19.184466+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:04.526204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.24","version":"33'4","reported_seq":32,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.028180+0000","last_change":"2024-09-17T04:34:00.134272+0000","last_active":"2024-09-17T04:34:20.028180+0000","last_peered":"2024-09-17T04:34:20.028180+0000","last_clean":"2024-09-17T04:34:20.028180+0000","last_became_active":"2024-09-17T04:34:00.133409+0000","last_became_peered":"2024-09-17T04:34:00.133409+0000","last_unstale":"2024-09-17T04:34:20.028180+0000","last_undegraded":"2024-09-17T04:34:20.028180+0000","last_fullsized":"2024-09-17T04:34:20.028180+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:17:46.794401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"33'6","reported_seq":34,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.028088+0000","last_change":"2024-09-17T04:34:00.136351+0000","last_active":"2024-09-17T04:34:20.028088+0000","last_peered":"2024-09-17T04:34:20.028088+0000","last_clean":"2024-09-17T04:34:20.028088+0000","last_became_active":"2024-09-17T04:34:00.136007+0000","last_became_peered":"2024-09-17T04:34:00.136007+0000","last_unstale":"2024-09-17T04:34:20.028088+0000","last_undegraded":"2024-09-17T04:34:20.028088+0000","last_fullsized":"2024-09-17T04:34:20.028088+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:09:37.721140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"33'15","reported_seq":42,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.185106+0000","last_change":"2024-09-17T04:34:00.147296+0000","last_active":"2024-09-17T04:34:19.185106+0000","last_peered":"2024-09-17T04:34:19.185106+0000","last_clean":"2024-09-17T04:34:19.185106+0000","last_became_active":"2024-09-17T04:34:00.146984+0000","last_became_peered":"2024-09-17T04:34:00.146984+0000","last_unstale":"2024-09-17T04:34:19.185106+0000","last_undegraded":"2024-09-17T04:34:19.185106+0000","last_fullsized":"2024-09-17T04:34:19.185106+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:28:28.233328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.27","version":"33'2","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027089+0000","last_change":"2024-09-17T04:34:00.134911+0000","last_active":"2024-09-17T04:34:20.027089+0000","last_peered":"2024-09-17T04:34:20.027089+0000","last_clean":"2024-09-17T04:34:20.027089+0000","last_became_active":"2024-09-17T04:34:00.133373+0000","last_became_peered":"2024-09-17T04:34:00.133373+0000","last_unstale":"2024-09-17T04:34:20.027089+0000","last_undegraded":"2024-09-17T04:34:20.027089+0000","last_fullsized":"2024-09-17T04:34:20.027089+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:41.467535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.28","version":"33'6","reported_seq":34,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.026580+0000","last_change":"2024-09-17T04:34:00.137953+0000","last_active":"2024-09-17T04:34:20.026580+0000","last_peered":"2024-09-17T04:34:20.026580+0000","last_clean":"2024-09-17T04:34:20.026580+0000","last_became_active":"2024-09-17T04:34:00.137577+0000","last_became_peered":"2024-09-17T04:34:00.137577+0000","last_unstale":"2024-09-17T04:34:20.026580+0000","last_undegraded":"2024-09-17T04:34:20.026580+0000","last_fullsized":"2024-09-17T04:34:20.026580+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:09.704990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.29","version":"33'8","reported_seq":38,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027135+0000","last_change":"2024-09-17T04:34:00.132040+0000","last_active":"2024-09-17T04:34:20.027135+0000","last_peered":"2024-09-17T04:34:20.027135+0000","last_clean":"2024-09-17T04:34:20.027135+0000","last_became_active":"2024-09-17T04:34:00.131717+0000","last_became_peered":"2024-09-17T04:34:00.131717+0000","last_unstale":"2024-09-17T04:34:20.027135+0000","last_undegraded":"2024-09-17T04:34:20.027135+0000","last_fullsized":"2024-09-17T04:34:20.027135+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:48.008056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"33'9","reported_seq":38,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.185245+0000","last_change":"2024-09-17T04:34:00.148488+0000","last_active":"2024-09-17T04:34:19.185245+0000","last_peered":"2024-09-17T04:34:19.185245+0000","last_clean":"2024-09-17T04:34:19.185245+0000","last_became_active":"2024-09-17T04:34:00.148176+0000","last_became_peered":"2024-09-17T04:34:00.148176+0000","last_unstale":"2024-09-17T04:34:19.185245+0000","last_undegraded":"2024-09-17T04:34:19.185245+0000","last_fullsized":"2024-09-17T04:34:19.185245+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:24:03.422429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2b","version":"33'11","reported_seq":40,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184412+0000","last_change":"2024-09-17T04:34:00.133295+0000","last_active":"2024-09-17T04:34:19.184412+0000","last_peered":"2024-09-17T04:34:19.184412+0000","last_clean":"2024-09-17T04:34:19.184412+0000","last_became_active":"2024-09-17T04:34:00.132965+0000","last_became_peered":"2024-09-17T04:34:00.132965+0000","last_unstale":"2024-09-17T04:34:19.184412+0000","last_undegraded":"2024-09-17T04:34:19.184412+0000","last_fullsized":"2024-09-17T04:34:19.184412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:29.313606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2c","version":"33'8","reported_seq":36,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.183342+0000","last_change":"2024-09-17T04:34:00.140053+0000","last_active":"2024-09-17T04:34:19.183342+0000","last_peered":"2024-09-17T04:34:19.183342+0000","last_clean":"2024-09-17T04:34:19.183342+0000","last_became_active":"2024-09-17T04:34:00.139557+0000","last_became_peered":"2024-09-17T04:34:00.139557+0000","last_unstale":"2024-09-17T04:34:19.183342+0000","last_undegraded":"2024-09-17T04:34:19.183342+0000","last_fullsized":"2024-09-17T04:34:19.183342+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:02:15.227761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2d","version":"33'7","reported_seq":34,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184495+0000","last_change":"2024-09-17T04:34:00.149546+0000","last_active":"2024-09-17T04:34:19.184495+0000","last_peered":"2024-09-17T04:34:19.184495+0000","last_clean":"2024-09-17T04:34:19.184495+0000","last_became_active":"2024-09-17T04:34:00.149166+0000","last_became_peered":"2024-09-17T04:34:00.149166+0000","last_unstale":"2024-09-17T04:34:19.184495+0000","last_undegraded":"2024-09-17T04:34:19.184495+0000","last_fullsized":"2024-09-17T04:34:19.184495+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:01.405073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"33'10","reported_seq":37,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.185020+0000","last_change":"2024-09-17T04:34:00.147773+0000","last_active":"2024-09-17T04:34:19.185020+0000","last_peered":"2024-09-17T04:34:19.185020+0000","last_clean":"2024-09-17T04:34:19.185020+0000","last_became_active":"2024-09-17T04:34:00.147077+0000","last_became_peered":"2024-09-17T04:34:00.147077+0000","last_unstale":"2024-09-17T04:34:19.185020+0000","last_undegraded":"2024-09-17T04:34:19.185020+0000","last_fullsized":"2024-09-17T04:34:19.185020+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:22:33.810133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2f","version":"33'9","reported_seq":39,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.028783+0000","last_change":"2024-09-17T04:34:00.134671+0000","last_active":"2024-09-17T04:34:20.028783+0000","last_peered":"2024-09-17T04:34:20.028783+0000","last_clean":"2024-09-17T04:34:20.028783+0000","last_became_active":"2024-09-17T04:34:00.132323+0000","last_became_peered":"2024-09-17T04:34:00.132323+0000","last_unstale":"2024-09-17T04:34:20.028783+0000","last_undegraded":"2024-09-17T04:34:20.028783+0000","last_fullsized":"2024-09-17T04:34:20.028783+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:21:34.320145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.30","version":"33'9","reported_seq":39,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027489+0000","last_change":"2024-09-17T04:34:00.147088+0000","last_active":"2024-09-17T04:34:20.027489+0000","last_peered":"2024-09-17T04:34:20.027489+0000","last_clean":"2024-09-17T04:34:20.027489+0000","last_became_active":"2024-09-17T04:34:00.146778+0000","last_became_peered":"2024-09-17T04:34:00.146778+0000","last_unstale":"2024-09-17T04:34:20.027489+0000","last_undegraded":"2024-09-17T04:34:20.027489+0000","last_fullsized":"2024-09-17T04:34:20.027489+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:33.775620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.31","version":"33'7","reported_seq":35,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027101+0000","last_change":"2024-09-17T04:34:00.135929+0000","last_active":"2024-09-17T04:34:20.027101+0000","last_peered":"2024-09-17T04:34:20.027101+0000","last_clean":"2024-09-17T04:34:20.027101+0000","last_became_active":"2024-09-17T04:34:00.135615+0000","last_became_peered":"2024-09-17T04:34:00.135615+0000","last_unstale":"2024-09-17T04:34:20.027101+0000","last_undegraded":"2024-09-17T04:34:20.027101+0000","last_fullsized":"2024-09-17T04:34:20.027101+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:04:29.231102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.32","version":"33'8","reported_seq":35,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184662+0000","last_change":"2024-09-17T04:34:00.140479+0000","last_active":"2024-09-17T04:34:19.184662+0000","last_peered":"2024-09-17T04:34:19.184662+0000","last_clean":"2024-09-17T04:34:19.184662+0000","last_became_active":"2024-09-17T04:34:00.140125+0000","last_became_peered":"2024-09-17T04:34:00.140125+0000","last_unstale":"2024-09-17T04:34:19.184662+0000","last_undegraded":"2024-09-17T04:34:19.184662+0000","last_fullsized":"2024-09-17T04:34:19.184662+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:20.470320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.33","version":"33'10","reported_seq":38,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.183155+0000","last_change":"2024-09-17T04:34:00.137663+0000","last_active":"2024-09-17T04:34:19.183155+0000","last_peered":"2024-09-17T04:34:19.183155+0000","last_clean":"2024-09-17T04:34:19.183155+0000","last_became_active":"2024-09-17T04:34:00.137306+0000","last_became_peered":"2024-09-17T04:34:00.137306+0000","last_unstale":"2024-09-17T04:34:19.183155+0000","last_undegraded":"2024-09-17T04:34:19.183155+0000","last_fullsized":"2024-09-17T04:34:19.183155+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:53.056546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.34","version":"33'7","reported_seq":35,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.028110+0000","last_change":"2024-09-17T04:34:00.144495+0000","last_active":"2024-09-17T04:34:20.028110+0000","last_peered":"2024-09-17T04:34:20.028110+0000","last_clean":"2024-09-17T04:34:20.028110+0000","last_became_active":"2024-09-17T04:34:00.144027+0000","last_became_peered":"2024-09-17T04:34:00.144027+0000","last_unstale":"2024-09-17T04:34:20.028110+0000","last_undegraded":"2024-09-17T04:34:20.028110+0000","last_fullsized":"2024-09-17T04:34:20.028110+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:54.247439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.35","version":"33'6","reported_seq":36,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.029004+0000","last_change":"2024-09-17T04:34:00.144759+0000","last_active":"2024-09-17T04:34:20.029004+0000","last_peered":"2024-09-17T04:34:20.029004+0000","last_clean":"2024-09-17T04:34:20.029004+0000","last_became_active":"2024-09-17T04:34:00.144463+0000","last_became_peered":"2024-09-17T04:34:00.144463+0000","last_unstale":"2024-09-17T04:34:20.029004+0000","last_undegraded":"2024-09-17T04:34:20.029004+0000","last_fullsized":"2024-09-17T04:34:20.029004+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:46:55.181012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.36","version":"33'15","reported_seq":43,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027615+0000","last_change":"2024-09-17T04:34:00.147614+0000","last_active":"2024-09-17T04:34:20.027615+0000","last_peered":"2024-09-17T04:34:20.027615+0000","last_clean":"2024-09-17T04:34:20.027615+0000","last_became_active":"2024-09-17T04:34:00.147198+0000","last_became_peered":"2024-09-17T04:34:00.147198+0000","last_unstale":"2024-09-17T04:34:20.027615+0000","last_undegraded":"2024-09-17T04:34:20.027615+0000","last_fullsized":"2024-09-17T04:34:20.027615+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:24:35.627644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.37","version":"33'11","reported_seq":40,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.185224+0000","last_change":"2024-09-17T04:34:00.126244+0000","last_active":"2024-09-17T04:34:19.185224+0000","last_peered":"2024-09-17T04:34:19.185224+0000","last_clean":"2024-09-17T04:34:19.185224+0000","last_became_active":"2024-09-17T04:34:00.125921+0000","last_became_peered":"2024-09-17T04:34:00.125921+0000","last_unstale":"2024-09-17T04:34:19.185224+0000","last_undegraded":"2024-09-17T04:34:19.185224+0000","last_fullsized":"2024-09-17T04:34:19.185224+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:05:45.109184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.38","version":"33'9","reported_seq":39,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027334+0000","last_change":"2024-09-17T04:34:00.136461+0000","last_active":"2024-09-17T04:34:20.027334+0000","last_peered":"2024-09-17T04:34:20.027334+0000","last_clean":"2024-09-17T04:34:20.027334+0000","last_became_active":"2024-09-17T04:34:00.136127+0000","last_became_peered":"2024-09-17T04:34:00.136127+0000","last_unstale":"2024-09-17T04:34:20.027334+0000","last_undegraded":"2024-09-17T04:34:20.027334+0000","last_fullsized":"2024-09-17T04:34:20.027334+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:24.656948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"33'5","reported_seq":35,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027496+0000","last_change":"2024-09-17T04:34:00.135963+0000","last_active":"2024-09-17T04:34:20.027496+0000","last_peered":"2024-09-17T04:34:20.027496+0000","last_clean":"2024-09-17T04:34:20.027496+0000","last_became_active":"2024-09-17T04:34:00.135466+0000","last_became_peered":"2024-09-17T04:34:00.135466+0000","last_unstale":"2024-09-17T04:34:20.027496+0000","last_undegraded":"2024-09-17T04:34:20.027496+0000","last_fullsized":"2024-09-17T04:34:20.027496+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:46:16.609038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"33'6","reported_seq":33,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184510+0000","last_change":"2024-09-17T04:34:00.139638+0000","last_active":"2024-09-17T04:34:19.184510+0000","last_peered":"2024-09-17T04:34:19.184510+0000","last_clean":"2024-09-17T04:34:19.184510+0000","last_became_active":"2024-09-17T04:34:00.139252+0000","last_became_peered":"2024-09-17T04:34:00.139252+0000","last_unstale":"2024-09-17T04:34:19.184510+0000","last_undegraded":"2024-09-17T04:34:19.184510+0000","last_fullsized":"2024-09-17T04:34:19.184510+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:10.516418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3b","version":"33'4","reported_seq":32,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.183018+0000","last_change":"2024-09-17T04:34:00.148227+0000","last_active":"2024-09-17T04:34:19.183018+0000","last_peered":"2024-09-17T04:34:19.183018+0000","last_clean":"2024-09-17T04:34:19.183018+0000","last_became_active":"2024-09-17T04:34:00.147902+0000","last_became_peered":"2024-09-17T04:34:00.147902+0000","last_unstale":"2024-09-17T04:34:19.183018+0000","last_undegraded":"2024-09-17T04:34:19.183018+0000","last_fullsized":"2024-09-17T04:34:19.183018+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:15.362788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3c","version":"33'8","reported_seq":36,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027748+0000","last_change":"2024-09-17T04:34:00.138845+0000","last_active":"2024-09-17T04:34:20.027748+0000","last_peered":"2024-09-17T04:34:20.027748+0000","last_clean":"2024-09-17T04:34:20.027748+0000","last_became_active":"2024-09-17T04:34:00.138517+0000","last_became_peered":"2024-09-17T04:34:00.138517+0000","last_unstale":"2024-09-17T04:34:20.027748+0000","last_undegraded":"2024-09-17T04:34:20.027748+0000","last_fullsized":"2024-09-17T04:34:20.027748+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:58.534910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3d","version":"33'7","reported_seq":35,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.028180+0000","last_change":"2024-09-17T04:34:00.146084+0000","last_active":"2024-09-17T04:34:20.028180+0000","last_peered":"2024-09-17T04:34:20.028180+0000","last_clean":"2024-09-17T04:34:20.028180+0000","last_became_active":"2024-09-17T04:34:00.145314+0000","last_became_peered":"2024-09-17T04:34:00.145314+0000","last_unstale":"2024-09-17T04:34:20.028180+0000","last_undegraded":"2024-09-17T04:34:20.028180+0000","last_fullsized":"2024-09-17T04:34:20.028180+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:10.516418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3e","version":"33'4","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:19.184980+0000","last_change":"2024-09-17T04:34:00.141408+0000","last_active":"2024-09-17T04:34:19.184980+0000","last_peered":"2024-09-17T04:34:19.184980+0000","last_clean":"2024-09-17T04:34:19.184980+0000","last_became_active":"2024-09-17T04:34:00.141077+0000","last_became_peered":"2024-09-17T04:34:00.141077+0000","last_unstale":"2024-09-17T04:34:19.184980+0000","last_undegraded":"2024-09-17T04:34:19.184980+0000","last_fullsized":"2024-09-17T04:34:19.184980+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:15:12.698992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"33'8","reported_seq":36,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-17T04:34:20.027531+0000","last_change":"2024-09-17T04:34:00.133489+0000","last_active":"2024-09-17T04:34:20.027531+0000","last_peered":"2024-09-17T04:34:20.027531+0000","last_clean":"2024-09-17T04:34:20.027531+0000","last_became_active":"2024-09-17T04:34:00.133172+0000","last_became_peered":"2024-09-17T04:34:00.133172+0000","last_unstale":"2024-09-17T04:34:20.027531+0000","last_undegraded":"2024-09-17T04:34:20.027531+0000","last_fullsized":"2024-09-17T04:34:20.027531+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:59.092019+0000","last_clean_scrub_stamp":"2024-09-17T04:33:59.092019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:05:34.986501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":504,"ondisk_log_size":504,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":97,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84,"ondisk_log_size":84,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542163,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30120,"kb_used_data":3084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341720,"statfs":{"total":96636764160,"available":96605921280,"internally_reserved":0,"allocated":3158016,"data_stored":2880626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542163,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29532,"kb_used_data":2504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342308,"statfs":{"total":96636764160,"available":96606523392,"internally_reserved":0,"allocated":2564096,"data_stored":2290258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542164,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":4552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":4661248,"data_stored":4387410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31644,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340196,"statfs":{"total":96636764160,"available":96604360704,"internally_reserved":0,"allocated":466944,"data_stored":193106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542164,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":466944,"data_stored":193106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542163,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":466944,"data_stored":193106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542162,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":4552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":4661248,"data_stored":4387410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542163,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28072,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343768,"statfs":{"total":96636764160,"available":96608018432,"internally_reserved":0,"allocated":1060864,"data_stored":783474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:34:22.655 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T04:34:22.655 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -s 2024-09-17T04:34:22.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.805+0000 7fe663a40640 1 -- 172.21.15.37:0/926218108 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe65c0cc3b0 msgr2=0x7fe65c0cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:22.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.805+0000 7fe663a40640 1 --2- 172.21.15.37:0/926218108 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe65c0cc3b0 0x7fe65c0cc7c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe654005fd0 tx=0x7fe65402a8b0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.805+0000 7fe663a40640 1 -- 172.21.15.37:0/926218108 shutdown_connections 2024-09-17T04:34:22.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.805+0000 7fe663a40640 1 --2- 172.21.15.37:0/926218108 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe65c0cc3b0 0x7fe65c0cc7c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.805+0000 7fe663a40640 1 --2- 172.21.15.37:0/926218108 >> v2:172.21.15.92:3300/0 conn(0x7fe65c0cf870 0x7fe65c0b65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.805+0000 7fe663a40640 1 -- 172.21.15.37:0/926218108 >> 172.21.15.37:0/926218108 conn(0x7fe65c0d13b0 msgr2=0x7fe65c0d17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:22.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.805+0000 7fe663a40640 1 -- 172.21.15.37:0/926218108 shutdown_connections 2024-09-17T04:34:22.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.805+0000 7fe663a40640 1 -- 172.21.15.37:0/926218108 wait complete. 2024-09-17T04:34:22.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.807+0000 7fe663a40640 1 Processor -- start 2024-09-17T04:34:22.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.807+0000 7fe663a40640 1 -- start start 2024-09-17T04:34:22.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.808+0000 7fe663a40640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe65c0b0ac0 0x7fe65c1f96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:22.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.808+0000 7fe663a40640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe65c0cc3b0 0x7fe65c146300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:22.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.808+0000 7fe663a40640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe65c0cf870 0x7fe65c0afa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:22.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.808+0000 7fe663a40640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c0cdac0 con 0x7fe65c0cc3b0 2024-09-17T04:34:22.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.808+0000 7fe663a40640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c0cd1e0 con 0x7fe65c0b0ac0 2024-09-17T04:34:22.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.808+0000 7fe663a40640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fe65c0cdc40 con 0x7fe65c0cf870 2024-09-17T04:34:22.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.808+0000 7fe660fdb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe65c0cc3b0 0x7fe65c146300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:22.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.808+0000 7fe660fdb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe65c0cc3b0 0x7fe65c146300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38600/0 (socket says 172.21.15.37:38600) 2024-09-17T04:34:22.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.808+0000 7fe660fdb640 1 -- 172.21.15.37:0/3381753574 learned_addr learned my addr 172.21.15.37:0/3381753574 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:22.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.809+0000 7fe6617dc640 1 --2- 172.21.15.37:0/3381753574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe65c0b0ac0 0x7fe65c1f96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:22.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.809+0000 7fe661fdd640 1 --2- 172.21.15.37:0/3381753574 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe65c0cf870 0x7fe65c0afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:22.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.809+0000 7fe660fdb640 1 -- 172.21.15.37:0/3381753574 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe65c0cf870 msgr2=0x7fe65c0afa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:22.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.809+0000 7fe660fdb640 1 --2- 172.21.15.37:0/3381753574 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe65c0cf870 0x7fe65c0afa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.809+0000 7fe660fdb640 1 -- 172.21.15.37:0/3381753574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe65c0b0ac0 msgr2=0x7fe65c1f96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:22.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.809+0000 7fe660fdb640 1 --2- 172.21.15.37:0/3381753574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe65c0b0ac0 0x7fe65c1f96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:22.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.809+0000 7fe660fdb640 1 -- 172.21.15.37:0/3381753574 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe65c15ef30 con 0x7fe65c0cc3b0 2024-09-17T04:34:22.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.809+0000 7fe660fdb640 1 --2- 172.21.15.37:0/3381753574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe65c0cc3b0 0x7fe65c146300 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00ea20 tx=0x7fe64c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:22.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.809+0000 7fe651ffb640 1 -- 172.21.15.37:0/3381753574 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c00cd80 con 0x7fe65c0cc3b0 2024-09-17T04:34:22.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.809+0000 7fe663a40640 1 -- 172.21.15.37:0/3381753574 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe65c15f210 con 0x7fe65c0cc3b0 2024-09-17T04:34:22.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.809+0000 7fe651ffb640 1 -- 172.21.15.37:0/3381753574 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe64c010430 con 0x7fe65c0cc3b0 2024-09-17T04:34:22.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.809+0000 7fe651ffb640 1 -- 172.21.15.37:0/3381753574 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c010880 con 0x7fe65c0cc3b0 2024-09-17T04:34:22.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.809+0000 7fe663a40640 1 -- 172.21.15.37:0/3381753574 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe65c0af520 con 0x7fe65c0cc3b0 2024-09-17T04:34:22.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.810+0000 7fe663a40640 1 -- 172.21.15.37:0/3381753574 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe65c0cc7c0 con 0x7fe65c0cc3b0 2024-09-17T04:34:22.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.811+0000 7fe651ffb640 1 -- 172.21.15.37:0/3381753574 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe64c010a20 con 0x7fe65c0cc3b0 2024-09-17T04:34:22.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.812+0000 7fe651ffb640 1 --2- 172.21.15.37:0/3381753574 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe640078b90 0x7fe64007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:22.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.812+0000 7fe651ffb640 1 -- 172.21.15.37:0/3381753574 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7fe64c09b990 con 0x7fe65c0cc3b0 2024-09-17T04:34:22.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.812+0000 7fe6617dc640 1 --2- 172.21.15.37:0/3381753574 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe640078b90 0x7fe64007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:22.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.813+0000 7fe6617dc640 1 --2- 172.21.15.37:0/3381753574 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe640078b90 0x7fe64007b0a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe658001720 tx=0x7fe658009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:22.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:22.816+0000 7fe651ffb640 1 -- 172.21.15.37:0/3381753574 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe64c0634d0 con 0x7fe65c0cc3b0 2024-09-17T04:34:23.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.077+0000 7fe663a40640 1 -- 172.21.15.37:0/3381753574 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fe65c15f3e0 con 0x7fe65c0cc3b0 2024-09-17T04:34:23.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.077+0000 7fe651ffb640 1 -- 172.21.15.37:0/3381753574 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+740 (secure 0 0 0) 0x7fe64c067f00 con 0x7fe65c0cc3b0 2024-09-17T04:34:23.077 INFO:teuthology.orchestra.run.smithi037.stdout: cluster: 2024-09-17T04:34:23.078 INFO:teuthology.orchestra.run.smithi037.stdout: id: a908a603-430c-4209-8294-0c3460b35f42 2024-09-17T04:34:23.078 INFO:teuthology.orchestra.run.smithi037.stdout: health: HEALTH_ERR 2024-09-17T04:34:23.078 INFO:teuthology.orchestra.run.smithi037.stdout: 2 filesystems are degraded 2024-09-17T04:34:23.078 INFO:teuthology.orchestra.run.smithi037.stdout: 2 filesystems are offline 2024-09-17T04:34:23.078 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:23.078 INFO:teuthology.orchestra.run.smithi037.stdout: services: 2024-09-17T04:34:23.078 INFO:teuthology.orchestra.run.smithi037.stdout: mon: 3 daemons, quorum a,b,c (age 88s) 2024-09-17T04:34:23.078 INFO:teuthology.orchestra.run.smithi037.stdout: mgr: x(active, since 84s), standbys: y 2024-09-17T04:34:23.078 INFO:teuthology.orchestra.run.smithi037.stdout: mds: 0/2 daemons up (2 failed), 4 standby 2024-09-17T04:34:23.078 INFO:teuthology.orchestra.run.smithi037.stdout: osd: 8 osds: 8 up (since 76s), 8 in (since 85s) 2024-09-17T04:34:23.078 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:23.078 INFO:teuthology.orchestra.run.smithi037.stdout: data: 2024-09-17T04:34:23.078 INFO:teuthology.orchestra.run.smithi037.stdout: volumes: 0/2 healthy, 2 failed 2024-09-17T04:34:23.078 INFO:teuthology.orchestra.run.smithi037.stdout: pools: 4 pools, 129 pgs 2024-09-17T04:34:23.079 INFO:teuthology.orchestra.run.smithi037.stdout: objects: 4 objects, 6.6 MiB 2024-09-17T04:34:23.079 INFO:teuthology.orchestra.run.smithi037.stdout: usage: 232 MiB used, 720 GiB / 720 GiB avail 2024-09-17T04:34:23.079 INFO:teuthology.orchestra.run.smithi037.stdout: pgs: 20.930% pgs unknown 2024-09-17T04:34:23.079 INFO:teuthology.orchestra.run.smithi037.stdout: 3.876% pgs not active 2024-09-17T04:34:23.079 INFO:teuthology.orchestra.run.smithi037.stdout: 97 active+clean 2024-09-17T04:34:23.079 INFO:teuthology.orchestra.run.smithi037.stdout: 27 unknown 2024-09-17T04:34:23.079 INFO:teuthology.orchestra.run.smithi037.stdout: 5 creating+peering 2024-09-17T04:34:23.079 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:23.080 INFO:teuthology.orchestra.run.smithi037.stdout: io: 2024-09-17T04:34:23.080 INFO:teuthology.orchestra.run.smithi037.stdout: client: 0 B/s rd, 2.3 KiB/s wr, 0 op/s rd, 10 op/s wr 2024-09-17T04:34:23.080 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:23.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.080+0000 7fe663a40640 1 -- 172.21.15.37:0/3381753574 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe640078b90 msgr2=0x7fe64007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.080+0000 7fe663a40640 1 --2- 172.21.15.37:0/3381753574 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe640078b90 0x7fe64007b0a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe658001720 tx=0x7fe658009040 comp rx=0 tx=0).stop 2024-09-17T04:34:23.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.080+0000 7fe663a40640 1 -- 172.21.15.37:0/3381753574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe65c0cc3b0 msgr2=0x7fe65c146300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.080+0000 7fe663a40640 1 --2- 172.21.15.37:0/3381753574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe65c0cc3b0 0x7fe65c146300 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00ea20 tx=0x7fe64c00eee0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.080+0000 7fe663a40640 1 -- 172.21.15.37:0/3381753574 shutdown_connections 2024-09-17T04:34:23.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.080+0000 7fe663a40640 1 --2- 172.21.15.37:0/3381753574 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe640078b90 0x7fe64007b0a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.080+0000 7fe663a40640 1 --2- 172.21.15.37:0/3381753574 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe65c0cf870 0x7fe65c0afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.080+0000 7fe663a40640 1 --2- 172.21.15.37:0/3381753574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe65c0cc3b0 0x7fe65c146300 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.080+0000 7fe663a40640 1 --2- 172.21.15.37:0/3381753574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe65c0b0ac0 0x7fe65c1f96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.080+0000 7fe663a40640 1 -- 172.21.15.37:0/3381753574 >> 172.21.15.37:0/3381753574 conn(0x7fe65c0d13b0 msgr2=0x7fe65c0709f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.080+0000 7fe663a40640 1 -- 172.21.15.37:0/3381753574 shutdown_connections 2024-09-17T04:34:23.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.080+0000 7fe663a40640 1 -- 172.21.15.37:0/3381753574 wait complete. 2024-09-17T04:34:23.091 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 init --force-init --filesystem recovery_fs --alternate-pool recovery_fs_metadata 2024-09-17T04:34:23.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.150+0000 7f5c5d20d740 1 -- start start 2024-09-17T04:34:23.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.150+0000 7f5c5d20d740 1 Processor -- start 2024-09-17T04:34:23.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.151+0000 7f5c5d20d740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56488cdcb680 0x56488d9f8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.151+0000 7f5c5d20d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56488cdcbb00 0x56488d9f8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.151+0000 7f5c5d20d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56488cdcbf80 0x56488d9f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.151+0000 7f5c5d20d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56488cd10780 con 0x56488cdcbb00 2024-09-17T04:34:23.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.151+0000 7f5c5d20d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56488cd10600 con 0x56488cdcbf80 2024-09-17T04:34:23.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.151+0000 7f5c5d20d740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x56488cd10480 con 0x56488cdcb680 2024-09-17T04:34:23.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.151+0000 7f5c5a6ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56488cdcbb00 0x56488d9f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.151+0000 7f5c5a6ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56488cdcbb00 0x56488d9f8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38602/0 (socket says 172.21.15.37:38602) 2024-09-17T04:34:23.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.151+0000 7f5c5a6ae640 1 -- 172.21.15.37:0/885401828 learned_addr learned my addr 172.21.15.37:0/885401828 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.151+0000 7f5c5b6b0640 1 --2- 172.21.15.37:0/885401828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56488cdcbf80 0x56488d9f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.151+0000 7f5c5aeaf640 1 --2- 172.21.15.37:0/885401828 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56488cdcb680 0x56488d9f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.151+0000 7f5c5a6ae640 1 -- 172.21.15.37:0/885401828 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56488cdcb680 msgr2=0x56488d9f8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.151+0000 7f5c5a6ae640 1 --2- 172.21.15.37:0/885401828 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56488cdcb680 0x56488d9f8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.151+0000 7f5c5a6ae640 1 -- 172.21.15.37:0/885401828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56488cdcbf80 msgr2=0x56488d9f8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.151+0000 7f5c5a6ae640 1 --2- 172.21.15.37:0/885401828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56488cdcbf80 0x56488d9f8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.151+0000 7f5c5a6ae640 1 -- 172.21.15.37:0/885401828 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564888c1f680 con 0x56488cdcbb00 2024-09-17T04:34:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.152+0000 7f5c5a6ae640 1 --2- 172.21.15.37:0/885401828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56488cdcbb00 0x56488d9f8000 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x56488dafe0f0 tx=0x56488cd26c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898288168b613299 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.152+0000 7f5c586aa640 1 -- 172.21.15.37:0/885401828 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564888c18ea0 con 0x56488cdcbb00 2024-09-17T04:34:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.152+0000 7f5c586aa640 1 -- 172.21.15.37:0/885401828 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56488cc788c0 con 0x56488cdcbb00 2024-09-17T04:34:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.152+0000 7f5c586aa640 1 -- 172.21.15.37:0/885401828 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564888c19380 con 0x56488cdcbb00 2024-09-17T04:34:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.152+0000 7f5c5d20d740 1 -- 172.21.15.37:0/885401828 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564888c1f4a0 con 0x56488cdcbb00 2024-09-17T04:34:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.152+0000 7f5c5d20d740 1 -- 172.21.15.37:0/885401828 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) -- 0x564888c1fa40 con 0x56488cdcbb00 2024-09-17T04:34:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.152+0000 7f5c5d20d740 4 waiting for MDS map... 2024-09-17T04:34:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.152+0000 7f5c586aa640 1 -- 172.21.15.37:0/885401828 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x56488db10600 con 0x56488cdcbb00 2024-09-17T04:34:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.152+0000 7f5c586aa640 1 -- 172.21.15.37:0/885401828 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 4143+0+0 (secure 0 0 0) 0x56488cc7d180 con 0x56488cdcbb00 2024-09-17T04:34:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.152+0000 7f5c5d20d740 4 Got MDS map 22 2024-09-17T04:34:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.152+0000 7f5c5d20d740 4 datascan.main: Using metadata pool output 2024-09-17T04:34:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.152+0000 7f5c5d20d740 4 datascan.main: connecting to RADOS... 2024-09-17T04:34:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.153+0000 7f5c5d20d740 1 Processor -- start 2024-09-17T04:34:23.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.153+0000 7f5c5d20d740 1 -- start start 2024-09-17T04:34:23.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.154+0000 7f5c5d20d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56488cdcad80 0x56488d9f9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.154+0000 7f5c5d20d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56488cdca900 0x56488d9f9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.154+0000 7f5c5d20d740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56488cdcc880 0x56488d9f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.154+0000 7f5c5d20d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56488cd10300 con 0x56488cdca900 2024-09-17T04:34:23.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.154+0000 7f5c5d20d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56488cd10000 con 0x56488cdcad80 2024-09-17T04:34:23.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.154+0000 7f5c5d20d740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x56488cc85800 con 0x56488cdcc880 2024-09-17T04:34:23.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.155+0000 7f5c5aeaf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56488cdca900 0x56488d9f9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.155+0000 7f5c5aeaf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56488cdca900 0x56488d9f9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38610/0 (socket says 172.21.15.37:38610) 2024-09-17T04:34:23.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.155+0000 7f5c5aeaf640 1 -- 172.21.15.37:0/4261422453 learned_addr learned my addr 172.21.15.37:0/4261422453 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.155+0000 7f5c5b6b0640 1 --2- 172.21.15.37:0/4261422453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56488cdcad80 0x56488d9f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.155+0000 7f5c5a6ae640 1 --2- 172.21.15.37:0/4261422453 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56488cdcc880 0x56488d9f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.155+0000 7f5c5aeaf640 1 -- 172.21.15.37:0/4261422453 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56488cdcc880 msgr2=0x56488d9f9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.155+0000 7f5c5aeaf640 1 --2- 172.21.15.37:0/4261422453 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56488cdcc880 0x56488d9f9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.155+0000 7f5c5aeaf640 1 -- 172.21.15.37:0/4261422453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56488cdcad80 msgr2=0x56488d9f9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.155+0000 7f5c5aeaf640 1 --2- 172.21.15.37:0/4261422453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56488cdcad80 0x56488d9f9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.155+0000 7f5c5aeaf640 1 -- 172.21.15.37:0/4261422453 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564888c1f860 con 0x56488cdca900 2024-09-17T04:34:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.155+0000 7f5c5b6b0640 1 --2- 172.21.15.37:0/4261422453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56488cdcad80 0x56488d9f9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.155+0000 7f5c5aeaf640 1 --2- 172.21.15.37:0/4261422453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56488cdca900 0x56488d9f9600 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x56488daff770 tx=0x56488db00c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6436b71f7787bca5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.155+0000 7f5c566a6640 1 -- 172.21.15.37:0/4261422453 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564888c18ea0 con 0x56488cdca900 2024-09-17T04:34:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.155+0000 7f5c566a6640 1 -- 172.21.15.37:0/4261422453 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56488cc78e00 con 0x56488cdca900 2024-09-17T04:34:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.155+0000 7f5c566a6640 1 -- 172.21.15.37:0/4261422453 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564888c19380 con 0x56488cdca900 2024-09-17T04:34:23.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.156+0000 7f5c5d20d740 1 -- 172.21.15.37:0/4261422453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56488cdca900 msgr2=0x56488d9f9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.156+0000 7f5c5d20d740 1 --2- 172.21.15.37:0/4261422453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56488cdca900 0x56488d9f9600 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x56488daff770 tx=0x56488db00c00 comp rx=0 tx=0).stop 2024-09-17T04:34:23.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.156+0000 7f5c5d20d740 1 -- 172.21.15.37:0/4261422453 shutdown_connections 2024-09-17T04:34:23.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.156+0000 7f5c5d20d740 1 --2- 172.21.15.37:0/4261422453 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56488cdcc880 0x56488d9f9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.156+0000 7f5c5d20d740 1 --2- 172.21.15.37:0/4261422453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56488cdca900 0x56488d9f9600 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.156+0000 7f5c5d20d740 1 --2- 172.21.15.37:0/4261422453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56488cdcad80 0x56488d9f9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.156+0000 7f5c5d20d740 1 -- 172.21.15.37:0/4261422453 >> 172.21.15.37:0/4261422453 conn(0x56488cdcb200 msgr2=0x56488cd45800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.156+0000 7f5c5d20d740 1 -- 172.21.15.37:0/4261422453 shutdown_connections 2024-09-17T04:34:23.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.156+0000 7f5c5d20d740 1 -- 172.21.15.37:0/4261422453 wait complete. 2024-09-17T04:34:23.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.157+0000 7f5c5d20d740 1 Processor -- start 2024-09-17T04:34:23.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.157+0000 7f5c5d20d740 1 -- start start 2024-09-17T04:34:23.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.158+0000 7f5c5d20d740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56488cdca900 0x56488d9f9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.158+0000 7f5c5d20d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56488cdcad80 0x56488d9f9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.158+0000 7f5c5d20d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56488cdcc880 0x56488d9f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.158+0000 7f5c5d20d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56488cd10000 con 0x56488cdcc880 2024-09-17T04:34:23.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.158+0000 7f5c5d20d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56488cd11800 con 0x56488cdcad80 2024-09-17T04:34:23.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.158+0000 7f5c5d20d740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x56488cd10300 con 0x56488cdca900 2024-09-17T04:34:23.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.158+0000 7f5c5a6ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56488cdcc880 0x56488d9f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.158+0000 7f5c5a6ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56488cdcc880 0x56488d9f9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38614/0 (socket says 172.21.15.37:38614) 2024-09-17T04:34:23.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.158+0000 7f5c5b6b0640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56488cdca900 0x56488d9f9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.158+0000 7f5c5a6ae640 1 -- 172.21.15.37:0/2591419498 learned_addr learned my addr 172.21.15.37:0/2591419498 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.158+0000 7f5c5aeaf640 1 --2- 172.21.15.37:0/2591419498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56488cdcad80 0x56488d9f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.158+0000 7f5c5a6ae640 1 -- 172.21.15.37:0/2591419498 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56488cdca900 msgr2=0x56488d9f9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.158+0000 7f5c5a6ae640 1 --2- 172.21.15.37:0/2591419498 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56488cdca900 0x56488d9f9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.158+0000 7f5c5a6ae640 1 -- 172.21.15.37:0/2591419498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56488cdcad80 msgr2=0x56488d9f9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.158+0000 7f5c5a6ae640 1 --2- 172.21.15.37:0/2591419498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56488cdcad80 0x56488d9f9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.158+0000 7f5c5a6ae640 1 -- 172.21.15.37:0/2591419498 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564888c1f680 con 0x56488cdcc880 2024-09-17T04:34:23.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.158+0000 7f5c5a6ae640 1 --2- 172.21.15.37:0/2591419498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56488cdcc880 0x56488d9f9b80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x56488daff620 tx=0x56488cd29140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.158+0000 7f5c54ea3640 1 -- 172.21.15.37:0/2591419498 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564888c191e0 con 0x56488cdcc880 2024-09-17T04:34:23.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.159+0000 7f5c54ea3640 1 -- 172.21.15.37:0/2591419498 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56488cc78fc0 con 0x56488cdcc880 2024-09-17T04:34:23.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.159+0000 7f5c5d20d740 1 -- 172.21.15.37:0/2591419498 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564888c1f4a0 con 0x56488cdcc880 2024-09-17T04:34:23.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.159+0000 7f5c54ea3640 1 -- 172.21.15.37:0/2591419498 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564888c18ea0 con 0x56488cdcc880 2024-09-17T04:34:23.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.159+0000 7f5c5d20d740 1 -- 172.21.15.37:0/2591419498 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564888c1fa40 con 0x56488cdcc880 2024-09-17T04:34:23.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.159+0000 7f5c5d20d740 4 datascan.init: forcing metadata pool 'recovery_fs_metadata' 2024-09-17T04:34:23.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.160+0000 7f5c54ea3640 1 -- 172.21.15.37:0/2591419498 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x56488db60900 con 0x56488cdcc880 2024-09-17T04:34:23.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.160+0000 7f5c54ea3640 1 --2- 172.21.15.37:0/2591419498 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x56488cdcda80 0x56488d9fa100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.160+0000 7f5c54ea3640 1 -- 172.21.15.37:0/2591419498 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x56488db82c00 con 0x56488cdcc880 2024-09-17T04:34:23.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.161+0000 7f5c5d20d740 1 --2- 172.21.15.37:0/2591419498 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x56488cdccd00 0x56488d9fa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.161+0000 7f5c5d20d740 1 -- 172.21.15.37:0/2591419498 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:1 6.14 6:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x56488db81c00 con 0x56488cdccd00 2024-09-17T04:34:23.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.161+0000 7f5c5b6b0640 1 --2- 172.21.15.37:0/2591419498 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x56488cdcda80 0x56488d9fa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.161+0000 7f5c5aeaf640 1 --2- 172.21.15.37:0/2591419498 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x56488cdccd00 0x56488d9fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.161+0000 7f5c5aeaf640 1 --2- 172.21.15.37:0/2591419498 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x56488cdccd00 0x56488d9fa680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.161+0000 7f5c5b6b0640 1 --2- 172.21.15.37:0/2591419498 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x56488cdcda80 0x56488d9fa100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x56488cdb9a70 tx=0x56488db01320 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.161+0000 7f5c5aeaf640 1 -- 172.21.15.37:0/2591419498 <== osd.2 v2:172.21.15.37:6816/579720176 1 ==== osd_op_reply(1 1.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 160+0+0 (crc 0 0 0) 0x56488cd22000 con 0x56488cdccd00 2024-09-17T04:34:23.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.161+0000 7f5c5d20d740 1 -- 172.21.15.37:0/2591419498 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:2 6.14 6:287034c4:::1.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x56488db80800 con 0x56488cdccd00 2024-09-17T04:34:23.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.163+0000 7f5c5aeaf640 1 -- 172.21.15.37:0/2591419498 <== osd.2 v2:172.21.15.37:6816/579720176 2 ==== osd_op_reply(2 1.00000000.inode [writefull 0~582] v41'1 uv1 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x56488cd22000 con 0x56488cdccd00 2024-09-17T04:34:23.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.163+0000 7f5c5d20d740 1 --2- 172.21.15.37:0/2591419498 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x56488cdcd180 0x56488d9fac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.163+0000 7f5c5d20d740 1 -- 172.21.15.37:0/2591419498 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:3 6.1f 6:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x56488db80000 con 0x56488cdcd180 2024-09-17T04:34:23.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.163+0000 7f5c5b6b0640 1 --2- 172.21.15.37:0/2591419498 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x56488cdcd180 0x56488d9fac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.164+0000 7f5c5b6b0640 1 --2- 172.21.15.37:0/2591419498 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x56488cdcd180 0x56488d9fac00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.164+0000 7f5c5b6b0640 1 -- 172.21.15.37:0/2591419498 <== osd.3 v2:172.21.15.37:6800/809975826 1 ==== osd_op_reply(3 100.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 162+0+0 (crc 0 0 0) 0x56488cd22500 con 0x56488cdcd180 2024-09-17T04:34:23.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.164+0000 7f5c5d20d740 1 -- 172.21.15.37:0/2591419498 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:4 6.1f 6:fe07bba1:::100.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x56488db80400 con 0x56488cdcd180 2024-09-17T04:34:23.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.166+0000 7f5c5b6b0640 1 -- 172.21.15.37:0/2591419498 <== osd.3 v2:172.21.15.37:6800/809975826 2 ==== osd_op_reply(4 100.00000000.inode [writefull 0~582] v41'1 uv1 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x56488cd22500 con 0x56488cdcd180 2024-09-17T04:34:23.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.166+0000 7f5c5d20d740 1 -- 172.21.15.37:0/2591419498 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:5 6.13 6:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x56488db81c00 con 0x56488cdcd180 2024-09-17T04:34:23.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.166+0000 7f5c5b6b0640 1 -- 172.21.15.37:0/2591419498 <== osd.3 v2:172.21.15.37:6800/809975826 3 ==== osd_op_reply(5 100.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56488cd22500 con 0x56488cdcd180 2024-09-17T04:34:23.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.166+0000 7f5c5d20d740 10 datascan.find_or_create_dirfrag: read_version = 0 2024-09-17T04:34:23.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.166+0000 7f5c5d20d740 1 -- 172.21.15.37:0/2591419498 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:6 6.13 6:cd5a64a3:::100.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x56488d042000 con 0x56488cdcd180 2024-09-17T04:34:23.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.167+0000 7f5c5b6b0640 1 -- 172.21.15.37:0/2591419498 <== osd.3 v2:172.21.15.37:6800/809975826 4 ==== osd_op_reply(6 100.00000000 [create,omap-set-header] v41'1 uv1 ondisk = 0) ==== 198+0+0 (crc 0 0 0) 0x56488cd22500 con 0x56488cdcd180 2024-09-17T04:34:23.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.167+0000 7f5c5d20d740 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x100 2024-09-17T04:34:23.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.167+0000 7f5c5d20d740 1 -- 172.21.15.37:0/2591419498 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x56488cdccd00 msgr2=0x56488d9fa680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.167+0000 7f5c5d20d740 1 --2- 172.21.15.37:0/2591419498 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x56488cdccd00 0x56488d9fa680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.167+0000 7f5c5d20d740 1 -- 172.21.15.37:0/2591419498 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x56488cdcd180 msgr2=0x56488d9fac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.167+0000 7f5c5d20d740 1 --2- 172.21.15.37:0/2591419498 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x56488cdcd180 0x56488d9fac00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.167+0000 7f5c5d20d740 1 -- 172.21.15.37:0/2591419498 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x56488cdcda80 msgr2=0x56488d9fa100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.167+0000 7f5c5d20d740 1 --2- 172.21.15.37:0/2591419498 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x56488cdcda80 0x56488d9fa100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x56488cdb9a70 tx=0x56488db01320 comp rx=0 tx=0).stop 2024-09-17T04:34:23.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.167+0000 7f5c5aeaf640 1 -- 172.21.15.37:0/2591419498 reap_dead start 2024-09-17T04:34:23.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.167+0000 7f5c5d20d740 1 -- 172.21.15.37:0/2591419498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56488cdcc880 msgr2=0x56488d9f9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.167+0000 7f5c5d20d740 1 --2- 172.21.15.37:0/2591419498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56488cdcc880 0x56488d9f9b80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x56488daff620 tx=0x56488cd29140 comp rx=0 tx=0).stop 2024-09-17T04:34:23.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.167+0000 7f5c5d20d740 1 -- 172.21.15.37:0/2591419498 shutdown_connections 2024-09-17T04:34:23.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.167+0000 7f5c5d20d740 1 --2- 172.21.15.37:0/2591419498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56488cdcc880 0x56488d9f9b80 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.167+0000 7f5c5d20d740 1 -- 172.21.15.37:0/2591419498 >> 172.21.15.37:0/2591419498 conn(0x56488cdcb200 msgr2=0x56488cd45800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.168+0000 7f5c5d20d740 1 -- 172.21.15.37:0/2591419498 shutdown_connections 2024-09-17T04:34:23.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.168+0000 7f5c5d20d740 1 -- 172.21.15.37:0/2591419498 wait complete. 2024-09-17T04:34:23.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.168+0000 7f5c5d20d740 1 -- 172.21.15.37:0/885401828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56488cdcbb00 msgr2=0x56488d9f8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.168+0000 7f5c5d20d740 1 --2- 172.21.15.37:0/885401828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56488cdcbb00 0x56488d9f8000 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x56488dafe0f0 tx=0x56488cd26c00 comp rx=0 tx=0).stop 2024-09-17T04:34:23.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.168+0000 7f5c5d20d740 1 -- 172.21.15.37:0/885401828 shutdown_connections 2024-09-17T04:34:23.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.168+0000 7f5c5d20d740 1 --2- 172.21.15.37:0/885401828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56488cdcbf80 0x56488d9f8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.168+0000 7f5c5d20d740 1 --2- 172.21.15.37:0/885401828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56488cdcbb00 0x56488d9f8000 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.168+0000 7f5c5d20d740 1 --2- 172.21.15.37:0/885401828 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56488cdcb680 0x56488d9f8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.168+0000 7f5c5d20d740 1 -- 172.21.15.37:0/885401828 >> 172.21.15.37:0/885401828 conn(0x56488cdcc400 msgr2=0x56488cd45000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.168+0000 7f5c5d20d740 1 -- 172.21.15.37:0/885401828 shutdown_connections 2024-09-17T04:34:23.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.168+0000 7f5c5d20d740 1 -- 172.21.15.37:0/885401828 wait complete. 2024-09-17T04:34:23.179 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'init', '--force-init', '--filesystem', 'recovery_fs', '--alternate-pool', 'recovery_fs_metadata'] in time 0:00:00.088202, result: 2024-09-17T04:34:23.179 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 recovery_fs:0 reset session 2024-09-17T04:34:23.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.232+0000 7f36204d3280 1 -- start start 2024-09-17T04:34:23.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.232+0000 7f36204d3280 1 Processor -- start 2024-09-17T04:34:23.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.233+0000 7f36204d3280 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e4055fa80 0x557e40486000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.233+0000 7f36204d3280 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x557e4055f180 0x557e40486580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.233+0000 7f36204d3280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e4119c000 0x557e40486b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.233+0000 7f36204d3280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557e404b4600 con 0x557e4119c000 2024-09-17T04:34:23.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.233+0000 7f36204d3280 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557e404b4480 con 0x557e4055fa80 2024-09-17T04:34:23.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.233+0000 7f36204d3280 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x557e404b4300 con 0x557e4055f180 2024-09-17T04:34:23.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.234+0000 7f361f47b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e4119c000 0x557e40486b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.234+0000 7f361f47b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e4119c000 0x557e40486b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38616/0 (socket says 172.21.15.37:38616) 2024-09-17T04:34:23.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.234+0000 7f361f47b640 1 -- 172.21.15.37:0/2726767737 learned_addr learned my addr 172.21.15.37:0/2726767737 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.234+0000 7f361e479640 1 --2- 172.21.15.37:0/2726767737 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x557e4055f180 0x557e40486580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.234+0000 7f361f47b640 1 -- 172.21.15.37:0/2726767737 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x557e4055f180 msgr2=0x557e40486580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.234+0000 7f361f47b640 1 --2- 172.21.15.37:0/2726767737 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x557e4055f180 0x557e40486580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.234+0000 7f361f47b640 1 -- 172.21.15.37:0/2726767737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e4055fa80 msgr2=0x557e40486000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.234+0000 7f361ec7a640 1 --2- 172.21.15.37:0/2726767737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e4055fa80 0x557e40486000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.234+0000 7f361f47b640 1 --2- 172.21.15.37:0/2726767737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e4055fa80 0x557e40486000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.234+0000 7f361f47b640 1 -- 172.21.15.37:0/2726767737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e40408780 con 0x557e4119c000 2024-09-17T04:34:23.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.234+0000 7f361ec7a640 1 --2- 172.21.15.37:0/2726767737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e4055fa80 0x557e40486000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:23.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.234+0000 7f361f47b640 1 --2- 172.21.15.37:0/2726767737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e4119c000 0x557e40486b00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x557e411f78c0 tx=0x557e40492720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5062a12f34fefda3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.234+0000 7f361c475640 1 -- 172.21.15.37:0/2726767737 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557e3c3c0ea0 con 0x557e4119c000 2024-09-17T04:34:23.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.234+0000 7f361c475640 1 -- 172.21.15.37:0/2726767737 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x557e3c3c6540 con 0x557e4119c000 2024-09-17T04:34:23.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.234+0000 7f361c475640 1 -- 172.21.15.37:0/2726767737 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557e3c3c1380 con 0x557e4119c000 2024-09-17T04:34:23.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.234+0000 7f36204d3280 1 -- 172.21.15.37:0/2726767737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e404083c0 con 0x557e4119c000 2024-09-17T04:34:23.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.234+0000 7f36204d3280 1 -- 172.21.15.37:0/2726767737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) -- 0x557e40409680 con 0x557e4119c000 2024-09-17T04:34:23.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.234+0000 7f36204d3280 4 waiting for MDS map... 2024-09-17T04:34:23.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.234+0000 7f361c475640 1 -- 172.21.15.37:0/2726767737 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x557e4129aa00 con 0x557e4119c000 2024-09-17T04:34:23.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.235+0000 7f361c475640 1 -- 172.21.15.37:0/2726767737 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 4143+0+0 (secure 0 0 0) 0x557e40425180 con 0x557e4119c000 2024-09-17T04:34:23.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.235+0000 7f36204d3280 4 Got MDS map 22 2024-09-17T04:34:23.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.235+0000 7f36204d3280 10 main: main 2024-09-17T04:34:23.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.235+0000 7f36204d3280 4 main: connecting to RADOS... 2024-09-17T04:34:23.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.236+0000 7f36204d3280 1 Processor -- start 2024-09-17T04:34:23.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.236+0000 7f36204d3280 1 -- start start 2024-09-17T04:34:23.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.237+0000 7f36204d3280 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x557e4119c900 0x557e40487080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.237+0000 7f36204d3280 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e4119cd80 0x557e40487600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.237+0000 7f36204d3280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e4119d200 0x557e40487b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.237+0000 7f36204d3280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557e404b4180 con 0x557e4119d200 2024-09-17T04:34:23.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.237+0000 7f36204d3280 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557e404b4900 con 0x557e4119cd80 2024-09-17T04:34:23.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.237+0000 7f36204d3280 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x557e4042d800 con 0x557e4119c900 2024-09-17T04:34:23.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.237+0000 7f361e479640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e4119d200 0x557e40487b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.237+0000 7f361e479640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e4119d200 0x557e40487b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38620/0 (socket says 172.21.15.37:38620) 2024-09-17T04:34:23.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.237+0000 7f361e479640 1 -- 172.21.15.37:0/4151675790 learned_addr learned my addr 172.21.15.37:0/4151675790 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.237+0000 7f361e479640 1 --2- 172.21.15.37:0/4151675790 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x557e4119c900 0x557e40487080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.237+0000 7f361ec7a640 1 --2- 172.21.15.37:0/4151675790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e4119cd80 0x557e40487600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.237+0000 7f361e479640 1 -- 172.21.15.37:0/4151675790 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x557e4119c900 msgr2=0x557e40487080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.237+0000 7f361e479640 1 --2- 172.21.15.37:0/4151675790 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x557e4119c900 0x557e40487080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.237+0000 7f361e479640 1 -- 172.21.15.37:0/4151675790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e4119cd80 msgr2=0x557e40487600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.237+0000 7f361e479640 1 --2- 172.21.15.37:0/4151675790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e4119cd80 0x557e40487600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.237+0000 7f361e479640 1 -- 172.21.15.37:0/4151675790 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e404094a0 con 0x557e4119d200 2024-09-17T04:34:23.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.237+0000 7f361e479640 1 --2- 172.21.15.37:0/4151675790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e4119d200 0x557e40487b80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x557e412b6c30 tx=0x557e412becc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bdc4e39e9b6fb0e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.237+0000 7f361a471640 1 -- 172.21.15.37:0/4151675790 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557e3c3c0ea0 con 0x557e4119d200 2024-09-17T04:34:23.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.238+0000 7f361a471640 1 -- 172.21.15.37:0/4151675790 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x557e3c3c6fc0 con 0x557e4119d200 2024-09-17T04:34:23.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.238+0000 7f361a471640 1 -- 172.21.15.37:0/4151675790 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557e3c3c1380 con 0x557e4119d200 2024-09-17T04:34:23.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.238+0000 7f36204d3280 1 -- 172.21.15.37:0/4151675790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e4119d200 msgr2=0x557e40487b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.238+0000 7f36204d3280 1 --2- 172.21.15.37:0/4151675790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e4119d200 0x557e40487b80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x557e412b6c30 tx=0x557e412becc0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.238+0000 7f36204d3280 1 -- 172.21.15.37:0/4151675790 shutdown_connections 2024-09-17T04:34:23.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.238+0000 7f36204d3280 1 --2- 172.21.15.37:0/4151675790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e4119d200 0x557e40487b80 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.238+0000 7f36204d3280 1 --2- 172.21.15.37:0/4151675790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e4119cd80 0x557e40487600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.238+0000 7f36204d3280 1 --2- 172.21.15.37:0/4151675790 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x557e4119c900 0x557e40487080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.238+0000 7f36204d3280 1 -- 172.21.15.37:0/4151675790 >> 172.21.15.37:0/4151675790 conn(0x557e4119c480 msgr2=0x557e404e5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.238+0000 7f36204d3280 1 -- 172.21.15.37:0/4151675790 shutdown_connections 2024-09-17T04:34:23.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.238+0000 7f36204d3280 1 -- 172.21.15.37:0/4151675790 wait complete. 2024-09-17T04:34:23.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.239+0000 7f36204d3280 1 Processor -- start 2024-09-17T04:34:23.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.239+0000 7f36204d3280 1 -- start start 2024-09-17T04:34:23.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.240+0000 7f36204d3280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e4119c900 0x557e40487080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.240+0000 7f36204d3280 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x557e4119cd80 0x557e40487600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.240+0000 7f36204d3280 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e4119d200 0x557e40487b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.240+0000 7f36204d3280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557e404b4900 con 0x557e4119c900 2024-09-17T04:34:23.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.240+0000 7f36204d3280 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557e404b5980 con 0x557e4119d200 2024-09-17T04:34:23.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.240+0000 7f36204d3280 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x557e404b4180 con 0x557e4119cd80 2024-09-17T04:34:23.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.240+0000 7f361e479640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e4119c900 0x557e40487080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.240+0000 7f361e479640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e4119c900 0x557e40487080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38630/0 (socket says 172.21.15.37:38630) 2024-09-17T04:34:23.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.240+0000 7f361e479640 1 -- 172.21.15.37:0/1471999418 learned_addr learned my addr 172.21.15.37:0/1471999418 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.241+0000 7f361ec7a640 1 --2- 172.21.15.37:0/1471999418 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x557e4119cd80 0x557e40487600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.241+0000 7f361e479640 1 --2- 172.21.15.37:0/1471999418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e4119d200 0x557e40487b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.241+0000 7f361e479640 1 -- 172.21.15.37:0/1471999418 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x557e4119cd80 msgr2=0x557e40487600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.241+0000 7f361e479640 1 --2- 172.21.15.37:0/1471999418 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x557e4119cd80 0x557e40487600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.241+0000 7f361e479640 1 -- 172.21.15.37:0/1471999418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e4119d200 msgr2=0x557e40487b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.241+0000 7f361e479640 1 --2- 172.21.15.37:0/1471999418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e4119d200 0x557e40487b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.241+0000 7f361e479640 1 -- 172.21.15.37:0/1471999418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e40408780 con 0x557e4119c900 2024-09-17T04:34:23.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.241+0000 7f361ec7a640 1 --2- 172.21.15.37:0/1471999418 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x557e4119cd80 0x557e40487600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:23.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.241+0000 7f361e479640 1 --2- 172.21.15.37:0/1471999418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e4119c900 0x557e40487080 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x557e412b78f0 tx=0x557e412bee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.241+0000 7f3618c6e640 1 -- 172.21.15.37:0/1471999418 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557e3c3c1040 con 0x557e4119c900 2024-09-17T04:34:23.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.241+0000 7f3618c6e640 1 -- 172.21.15.37:0/1471999418 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x557e3c3c6e00 con 0x557e4119c900 2024-09-17T04:34:23.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.241+0000 7f36204d3280 1 -- 172.21.15.37:0/1471999418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557e404083c0 con 0x557e4119c900 2024-09-17T04:34:23.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.241+0000 7f3618c6e640 1 -- 172.21.15.37:0/1471999418 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557e3c3c0ea0 con 0x557e4119c900 2024-09-17T04:34:23.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.241+0000 7f36204d3280 1 -- 172.21.15.37:0/1471999418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e40409680 con 0x557e4119c900 2024-09-17T04:34:23.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.241+0000 7f36204d3280 4 main: resolving pool 6 2024-09-17T04:34:23.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.243+0000 7f3618c6e640 1 -- 172.21.15.37:0/1471999418 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x557e41304900 con 0x557e4119c900 2024-09-17T04:34:23.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.243+0000 7f3618c6e640 1 --2- 172.21.15.37:0/1471999418 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x557e4119f600 0x557e40488100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.243+0000 7f3618c6e640 1 -- 172.21.15.37:0/1471999418 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x557e411d0a00 con 0x557e4119c900 2024-09-17T04:34:23.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.243+0000 7f36204d3280 4 main: creating IoCtx.. 2024-09-17T04:34:23.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.243+0000 7f36204d3280 1 --2- 172.21.15.37:0/1471999418 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x557e4119d680 0x557e40488680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.243+0000 7f36204d3280 1 -- 172.21.15.37:0/1471999418 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:1 6.b 6:d0630e4c:::mds0_sessionmap:head [omap-clear,omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x557e41326c00 con 0x557e4119d680 2024-09-17T04:34:23.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.243+0000 7f361ec7a640 1 --2- 172.21.15.37:0/1471999418 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x557e4119f600 0x557e40488100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.243+0000 7f361e479640 1 --2- 172.21.15.37:0/1471999418 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x557e4119d680 0x557e40488680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.244+0000 7f361e479640 1 --2- 172.21.15.37:0/1471999418 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x557e4119d680 0x557e40488680 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.244+0000 7f361ec7a640 1 --2- 172.21.15.37:0/1471999418 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x557e4119f600 0x557e40488100 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x557e40490300 tx=0x557e407d0000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.245+0000 7f361e479640 1 -- 172.21.15.37:0/1471999418 <== osd.3 v2:172.21.15.37:6800/809975826 1 ==== osd_op_reply(1 mds0_sessionmap [omap-clear,omap-set-header] v41'1 uv1 ondisk = 0) ==== 201+0+0 (crc 0 0 0) 0x557e404c2000 con 0x557e4119d680 2024-09-17T04:34:23.251 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T04:34:23.252 INFO:teuthology.orchestra.run.smithi037.stdout: "0": { 2024-09-17T04:34:23.252 INFO:teuthology.orchestra.run.smithi037.stdout: "data": {}, 2024-09-17T04:34:23.252 INFO:teuthology.orchestra.run.smithi037.stdout: "result": 0 2024-09-17T04:34:23.252 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:34:23.252 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T04:34:23.252 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:23.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.245+0000 7f36204d3280 1 -- 172.21.15.37:0/1471999418 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x557e4119d680 msgr2=0x557e40488680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.245+0000 7f36204d3280 1 --2- 172.21.15.37:0/1471999418 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x557e4119d680 0x557e40488680 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.245+0000 7f36204d3280 1 -- 172.21.15.37:0/1471999418 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x557e4119f600 msgr2=0x557e40488100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.245+0000 7f36204d3280 1 --2- 172.21.15.37:0/1471999418 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x557e4119f600 0x557e40488100 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x557e40490300 tx=0x557e407d0000 comp rx=0 tx=0).stop 2024-09-17T04:34:23.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.245+0000 7f36204d3280 1 -- 172.21.15.37:0/1471999418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e4119c900 msgr2=0x557e40487080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.245+0000 7f36204d3280 1 --2- 172.21.15.37:0/1471999418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e4119c900 0x557e40487080 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x557e412b78f0 tx=0x557e412bee40 comp rx=0 tx=0).stop 2024-09-17T04:34:23.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.245+0000 7f361ec7a640 1 -- 172.21.15.37:0/1471999418 reap_dead start 2024-09-17T04:34:23.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.245+0000 7f36204d3280 1 -- 172.21.15.37:0/1471999418 shutdown_connections 2024-09-17T04:34:23.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.245+0000 7f36204d3280 1 -- 172.21.15.37:0/1471999418 >> 172.21.15.37:0/1471999418 conn(0x557e4119c480 msgr2=0x557e404e5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.245+0000 7f36204d3280 1 -- 172.21.15.37:0/1471999418 shutdown_connections 2024-09-17T04:34:23.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.245+0000 7f36204d3280 1 -- 172.21.15.37:0/1471999418 wait complete. 2024-09-17T04:34:23.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.246+0000 7f36204d3280 1 -- 172.21.15.37:0/2726767737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e4119c000 msgr2=0x557e40486b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.246+0000 7f36204d3280 1 --2- 172.21.15.37:0/2726767737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e4119c000 0x557e40486b00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x557e411f78c0 tx=0x557e40492720 comp rx=0 tx=0).stop 2024-09-17T04:34:23.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.246+0000 7f36204d3280 1 -- 172.21.15.37:0/2726767737 shutdown_connections 2024-09-17T04:34:23.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.246+0000 7f36204d3280 1 --2- 172.21.15.37:0/2726767737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e4119c000 0x557e40486b00 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.246+0000 7f36204d3280 1 --2- 172.21.15.37:0/2726767737 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x557e4055f180 0x557e40486580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.246+0000 7f36204d3280 1 --2- 172.21.15.37:0/2726767737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557e4055fa80 0x557e40486000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.246+0000 7f36204d3280 1 -- 172.21.15.37:0/2726767737 >> 172.21.15.37:0/2726767737 conn(0x557e4055f600 msgr2=0x557e404e5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.246+0000 7f36204d3280 1 -- 172.21.15.37:0/2726767737 shutdown_connections 2024-09-17T04:34:23.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.246+0000 7f36204d3280 1 -- 172.21.15.37:0/2726767737 wait complete. 2024-09-17T04:34:23.257 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'recovery_fs:0', 'reset', 'session'] in time 0:00:00.077921, result: { "0": { "data": {}, "result": 0 } } 2024-09-17T04:34:23.258 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 recovery_fs:0 reset snap 2024-09-17T04:34:23.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.310+0000 7fc10d12a280 1 -- start start 2024-09-17T04:34:23.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.310+0000 7fc10d12a280 1 Processor -- start 2024-09-17T04:34:23.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.311+0000 7fc10d12a280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d49365a80 0x555d49fa6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.311+0000 7fc10d12a280 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d49365180 0x555d49fa6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.311+0000 7fc10d12a280 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x555d4a09e000 0x555d49fa6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.311+0000 7fc10d12a280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555d49fac180 con 0x555d49365a80 2024-09-17T04:34:23.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.312+0000 7fc10d12a280 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555d49fac300 con 0x555d49365180 2024-09-17T04:34:23.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.312+0000 7fc10d12a280 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x555d49fac480 con 0x555d4a09e000 2024-09-17T04:34:23.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.312+0000 7fc10b8cf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d49365a80 0x555d49fa6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.312+0000 7fc10b8cf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d49365a80 0x555d49fa6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38646/0 (socket says 172.21.15.37:38646) 2024-09-17T04:34:23.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.312+0000 7fc10b8cf640 1 -- 172.21.15.37:0/3606246594 learned_addr learned my addr 172.21.15.37:0/3606246594 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.312+0000 7fc10b0ce640 1 --2- 172.21.15.37:0/3606246594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d49365180 0x555d49fa6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.312+0000 7fc10b8cf640 1 -- 172.21.15.37:0/3606246594 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x555d4a09e000 msgr2=0x555d49fa6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.312+0000 7fc10c0d0640 1 --2- 172.21.15.37:0/3606246594 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x555d4a09e000 0x555d49fa6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.312+0000 7fc10b8cf640 1 --2- 172.21.15.37:0/3606246594 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x555d4a09e000 0x555d49fa6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.312+0000 7fc10b8cf640 1 -- 172.21.15.37:0/3606246594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d49365180 msgr2=0x555d49fa6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.312+0000 7fc10b8cf640 1 --2- 172.21.15.37:0/3606246594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d49365180 0x555d49fa6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.312+0000 7fc10b8cf640 1 -- 172.21.15.37:0/3606246594 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d4920e780 con 0x555d49365a80 2024-09-17T04:34:23.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.312+0000 7fc10b0ce640 1 --2- 172.21.15.37:0/3606246594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d49365180 0x555d49fa6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:23.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.312+0000 7fc10b8cf640 1 --2- 172.21.15.37:0/3606246594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d49365a80 0x555d49fa6000 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x555d49ff98c0 tx=0x555d492ccc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a74a21934aaf932 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.312+0000 7fc1090ca640 1 -- 172.21.15.37:0/3606246594 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555d451c6ea0 con 0x555d49365a80 2024-09-17T04:34:23.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.312+0000 7fc1090ca640 1 -- 172.21.15.37:0/3606246594 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x555d451cc380 con 0x555d49365a80 2024-09-17T04:34:23.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.312+0000 7fc1090ca640 1 -- 172.21.15.37:0/3606246594 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555d451c7380 con 0x555d49365a80 2024-09-17T04:34:23.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.312+0000 7fc10d12a280 1 -- 172.21.15.37:0/3606246594 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d4920e3c0 con 0x555d49365a80 2024-09-17T04:34:23.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.312+0000 7fc10d12a280 1 -- 172.21.15.37:0/3606246594 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) -- 0x555d4920f680 con 0x555d49365a80 2024-09-17T04:34:23.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.312+0000 7fc10d12a280 4 waiting for MDS map... 2024-09-17T04:34:23.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.313+0000 7fc1090ca640 1 -- 172.21.15.37:0/3606246594 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x555d4a0aea00 con 0x555d49365a80 2024-09-17T04:34:23.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.313+0000 7fc1090ca640 1 -- 172.21.15.37:0/3606246594 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 4143+0+0 (secure 0 0 0) 0x555d4922b180 con 0x555d49365a80 2024-09-17T04:34:23.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.313+0000 7fc10d12a280 4 Got MDS map 22 2024-09-17T04:34:23.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.313+0000 7fc10d12a280 10 main: main 2024-09-17T04:34:23.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.313+0000 7fc10d12a280 4 main: connecting to RADOS... 2024-09-17T04:34:23.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.314+0000 7fc10d12a280 1 Processor -- start 2024-09-17T04:34:23.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.314+0000 7fc10d12a280 1 -- start start 2024-09-17T04:34:23.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.315+0000 7fc10d12a280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d4a09e900 0x555d49fa7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.315+0000 7fc10d12a280 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d4a09ed80 0x555d49fa7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.315+0000 7fc10d12a280 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x555d4a09f200 0x555d49fa7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.315+0000 7fc10d12a280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555d49fac600 con 0x555d4a09e900 2024-09-17T04:34:23.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.315+0000 7fc10d12a280 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555d49fac780 con 0x555d4a09ed80 2024-09-17T04:34:23.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.315+0000 7fc10d12a280 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x555d49fac900 con 0x555d4a09f200 2024-09-17T04:34:23.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.315+0000 7fc10c0d0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d4a09e900 0x555d49fa7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.315+0000 7fc10c0d0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d4a09e900 0x555d49fa7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38650/0 (socket says 172.21.15.37:38650) 2024-09-17T04:34:23.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.315+0000 7fc10c0d0640 1 -- 172.21.15.37:0/515463404 learned_addr learned my addr 172.21.15.37:0/515463404 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.315+0000 7fc10b0ce640 1 --2- 172.21.15.37:0/515463404 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x555d4a09f200 0x555d49fa7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.315+0000 7fc10b8cf640 1 --2- 172.21.15.37:0/515463404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d4a09ed80 0x555d49fa7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.315+0000 7fc10c0d0640 1 -- 172.21.15.37:0/515463404 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x555d4a09f200 msgr2=0x555d49fa7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.315+0000 7fc10c0d0640 1 --2- 172.21.15.37:0/515463404 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x555d4a09f200 0x555d49fa7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.315+0000 7fc10c0d0640 1 -- 172.21.15.37:0/515463404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d4a09ed80 msgr2=0x555d49fa7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.315+0000 7fc10c0d0640 1 --2- 172.21.15.37:0/515463404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d4a09ed80 0x555d49fa7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.315+0000 7fc10c0d0640 1 -- 172.21.15.37:0/515463404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d4920f4a0 con 0x555d4a09e900 2024-09-17T04:34:23.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.316+0000 7fc10c0d0640 1 --2- 172.21.15.37:0/515463404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d4a09e900 0x555d49fa7080 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x555d4a0cccf0 tx=0x555d4a1240c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512760d71bf15cfd server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.316+0000 7fc1070c6640 1 -- 172.21.15.37:0/515463404 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555d451c6ea0 con 0x555d4a09e900 2024-09-17T04:34:23.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.316+0000 7fc1070c6640 1 -- 172.21.15.37:0/515463404 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x555d451cd180 con 0x555d4a09e900 2024-09-17T04:34:23.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.316+0000 7fc1070c6640 1 -- 172.21.15.37:0/515463404 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555d451c7380 con 0x555d4a09e900 2024-09-17T04:34:23.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.316+0000 7fc10d12a280 1 -- 172.21.15.37:0/515463404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d4a09e900 msgr2=0x555d49fa7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.316+0000 7fc10d12a280 1 --2- 172.21.15.37:0/515463404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d4a09e900 0x555d49fa7080 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x555d4a0cccf0 tx=0x555d4a1240c0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.316+0000 7fc10d12a280 1 -- 172.21.15.37:0/515463404 shutdown_connections 2024-09-17T04:34:23.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.316+0000 7fc10d12a280 1 --2- 172.21.15.37:0/515463404 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x555d4a09f200 0x555d49fa7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.316+0000 7fc10d12a280 1 --2- 172.21.15.37:0/515463404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d4a09ed80 0x555d49fa7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.316+0000 7fc10d12a280 1 --2- 172.21.15.37:0/515463404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d4a09e900 0x555d49fa7080 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.316+0000 7fc10d12a280 1 -- 172.21.15.37:0/515463404 >> 172.21.15.37:0/515463404 conn(0x555d4a09e480 msgr2=0x555d492eb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.316+0000 7fc10d12a280 1 -- 172.21.15.37:0/515463404 shutdown_connections 2024-09-17T04:34:23.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.316+0000 7fc10d12a280 1 -- 172.21.15.37:0/515463404 wait complete. 2024-09-17T04:34:23.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.317+0000 7fc10d12a280 1 Processor -- start 2024-09-17T04:34:23.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.317+0000 7fc10d12a280 1 -- start start 2024-09-17T04:34:23.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.318+0000 7fc10d12a280 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x555d4a09e900 0x555d49fa7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.318+0000 7fc10d12a280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d4a09ed80 0x555d49fa7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.318+0000 7fc10d12a280 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d4a09f200 0x555d49fa7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.318+0000 7fc10d12a280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555d49facd80 con 0x555d4a09ed80 2024-09-17T04:34:23.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.318+0000 7fc10d12a280 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555d49facf00 con 0x555d4a09f200 2024-09-17T04:34:23.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.318+0000 7fc10d12a280 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x555d49fad080 con 0x555d4a09e900 2024-09-17T04:34:23.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.319+0000 7fc10b8cf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d4a09ed80 0x555d49fa7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.319+0000 7fc10b8cf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d4a09ed80 0x555d49fa7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38658/0 (socket says 172.21.15.37:38658) 2024-09-17T04:34:23.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.319+0000 7fc10b8cf640 1 -- 172.21.15.37:0/863744987 learned_addr learned my addr 172.21.15.37:0/863744987 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.319+0000 7fc10c0d0640 1 --2- 172.21.15.37:0/863744987 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x555d4a09e900 0x555d49fa7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.319+0000 7fc10b0ce640 1 --2- 172.21.15.37:0/863744987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d4a09f200 0x555d49fa7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.319+0000 7fc10b8cf640 1 -- 172.21.15.37:0/863744987 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x555d4a09e900 msgr2=0x555d49fa7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.319+0000 7fc10b8cf640 1 --2- 172.21.15.37:0/863744987 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x555d4a09e900 0x555d49fa7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.319+0000 7fc10b8cf640 1 -- 172.21.15.37:0/863744987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d4a09f200 msgr2=0x555d49fa7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.319+0000 7fc10b8cf640 1 --2- 172.21.15.37:0/863744987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d4a09f200 0x555d49fa7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.319+0000 7fc10b8cf640 1 -- 172.21.15.37:0/863744987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d4920e780 con 0x555d4a09ed80 2024-09-17T04:34:23.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.319+0000 7fc10b0ce640 1 --2- 172.21.15.37:0/863744987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d4a09f200 0x555d49fa7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:23.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.319+0000 7fc10b8cf640 1 --2- 172.21.15.37:0/863744987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d4a09ed80 0x555d49fa7600 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x555d4a0cd9b0 tx=0x555d492ce8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.319+0000 7fc1058c3640 1 -- 172.21.15.37:0/863744987 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555d451c71e0 con 0x555d4a09ed80 2024-09-17T04:34:23.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.319+0000 7fc1058c3640 1 -- 172.21.15.37:0/863744987 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x555d451ccfc0 con 0x555d4a09ed80 2024-09-17T04:34:23.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.319+0000 7fc10d12a280 1 -- 172.21.15.37:0/863744987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555d4920e3c0 con 0x555d4a09ed80 2024-09-17T04:34:23.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.319+0000 7fc1058c3640 1 -- 172.21.15.37:0/863744987 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555d451c6ea0 con 0x555d4a09ed80 2024-09-17T04:34:23.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.319+0000 7fc10d12a280 1 -- 172.21.15.37:0/863744987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d4920f680 con 0x555d4a09ed80 2024-09-17T04:34:23.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.319+0000 7fc10d12a280 4 main: resolving pool 6 2024-09-17T04:34:23.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.321+0000 7fc1058c3640 1 -- 172.21.15.37:0/863744987 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x555d4a118900 con 0x555d4a09ed80 2024-09-17T04:34:23.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.321+0000 7fc1058c3640 1 --2- 172.21.15.37:0/863744987 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x555d4a0a1600 0x555d49fa8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.321+0000 7fc1058c3640 1 -- 172.21.15.37:0/863744987 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x555d49fd8c00 con 0x555d4a09ed80 2024-09-17T04:34:23.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.321+0000 7fc10d12a280 4 main: creating IoCtx.. 2024-09-17T04:34:23.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.321+0000 7fc10d12a280 1 --2- 172.21.15.37:0/863744987 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x555d4a09f680 0x555d49fa8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.321+0000 7fc10d12a280 1 -- 172.21.15.37:0/863744987 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:1 6.d 6:b50e409b:::mds_snaptable:head [writefull 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x555d49fd3c00 con 0x555d4a09f680 2024-09-17T04:34:23.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.321+0000 7fc10c0d0640 1 --2- 172.21.15.37:0/863744987 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x555d4a0a1600 0x555d49fa8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.322+0000 7fc10b0ce640 1 --2- 172.21.15.37:0/863744987 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x555d4a09f680 0x555d49fa8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.322+0000 7fc10c0d0640 1 --2- 172.21.15.37:0/863744987 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x555d4a0a1600 0x555d49fa8100 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x555d4a0cccf0 tx=0x555d4a1242a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.322+0000 7fc10b0ce640 1 --2- 172.21.15.37:0/863744987 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x555d4a09f680 0x555d49fa8680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.324+0000 7fc10b0ce640 1 -- 172.21.15.37:0/863744987 <== osd.5 v2:172.21.15.92:6826/248263832 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~70] v41'1 uv1 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x555d492c8000 con 0x555d4a09f680 2024-09-17T04:34:23.328 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T04:34:23.328 INFO:teuthology.orchestra.run.smithi037.stdout: "result": 0 2024-09-17T04:34:23.328 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T04:34:23.328 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:23.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.324+0000 7fc10d12a280 1 -- 172.21.15.37:0/863744987 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x555d4a09f680 msgr2=0x555d49fa8680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.324+0000 7fc10d12a280 1 --2- 172.21.15.37:0/863744987 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x555d4a09f680 0x555d49fa8680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.324+0000 7fc10d12a280 1 -- 172.21.15.37:0/863744987 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x555d4a0a1600 msgr2=0x555d49fa8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.324+0000 7fc10d12a280 1 --2- 172.21.15.37:0/863744987 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x555d4a0a1600 0x555d49fa8100 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x555d4a0cccf0 tx=0x555d4a1242a0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.324+0000 7fc10d12a280 1 -- 172.21.15.37:0/863744987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d4a09ed80 msgr2=0x555d49fa7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.324+0000 7fc10d12a280 1 --2- 172.21.15.37:0/863744987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d4a09ed80 0x555d49fa7600 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x555d4a0cd9b0 tx=0x555d492ce8a0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.324+0000 7fc10b0ce640 1 -- 172.21.15.37:0/863744987 reap_dead start 2024-09-17T04:34:23.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.324+0000 7fc10d12a280 1 -- 172.21.15.37:0/863744987 shutdown_connections 2024-09-17T04:34:23.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.324+0000 7fc10d12a280 1 -- 172.21.15.37:0/863744987 >> 172.21.15.37:0/863744987 conn(0x555d4a09e480 msgr2=0x555d492eb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.324+0000 7fc10d12a280 1 -- 172.21.15.37:0/863744987 shutdown_connections 2024-09-17T04:34:23.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.324+0000 7fc10d12a280 1 -- 172.21.15.37:0/863744987 wait complete. 2024-09-17T04:34:23.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.324+0000 7fc10d12a280 1 -- 172.21.15.37:0/3606246594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d49365a80 msgr2=0x555d49fa6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.324+0000 7fc10d12a280 1 --2- 172.21.15.37:0/3606246594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d49365a80 0x555d49fa6000 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x555d49ff98c0 tx=0x555d492ccc00 comp rx=0 tx=0).stop 2024-09-17T04:34:23.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.324+0000 7fc10d12a280 1 -- 172.21.15.37:0/3606246594 shutdown_connections 2024-09-17T04:34:23.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.324+0000 7fc10d12a280 1 --2- 172.21.15.37:0/3606246594 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x555d4a09e000 0x555d49fa6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.324+0000 7fc10d12a280 1 --2- 172.21.15.37:0/3606246594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555d49365180 0x555d49fa6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.324+0000 7fc10d12a280 1 --2- 172.21.15.37:0/3606246594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d49365a80 0x555d49fa6000 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.324+0000 7fc10d12a280 1 -- 172.21.15.37:0/3606246594 >> 172.21.15.37:0/3606246594 conn(0x555d49365600 msgr2=0x555d492eb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.325+0000 7fc10d12a280 1 -- 172.21.15.37:0/3606246594 shutdown_connections 2024-09-17T04:34:23.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.325+0000 7fc10d12a280 1 -- 172.21.15.37:0/3606246594 wait complete. 2024-09-17T04:34:23.332 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'recovery_fs:0', 'reset', 'snap'] in time 0:00:00.074346, result: { "result": 0 } 2024-09-17T04:34:23.332 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 recovery_fs:0 reset inode 2024-09-17T04:34:23.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.384+0000 7f0a2df99280 1 -- start start 2024-09-17T04:34:23.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.384+0000 7f0a2df99280 1 Processor -- start 2024-09-17T04:34:23.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.385+0000 7f0a2df99280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616168dba80 0x561617518000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.385+0000 7f0a2df99280 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5616168db180 0x561617518580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.385+0000 7f0a2df99280 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561617616000 0x561617518b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.385+0000 7f0a2df99280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561617532180 con 0x5616168dba80 2024-09-17T04:34:23.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.385+0000 7f0a2df99280 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561617532300 con 0x561617616000 2024-09-17T04:34:23.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.385+0000 7f0a2df99280 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x561617532480 con 0x5616168db180 2024-09-17T04:34:23.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.386+0000 7f0a2c740640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616168dba80 0x561617518000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.386+0000 7f0a2c740640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616168dba80 0x561617518000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38674/0 (socket says 172.21.15.37:38674) 2024-09-17T04:34:23.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.386+0000 7f0a2c740640 1 -- 172.21.15.37:0/3485125191 learned_addr learned my addr 172.21.15.37:0/3485125191 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.386+0000 7f0a2bf3f640 1 --2- 172.21.15.37:0/3485125191 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5616168db180 0x561617518580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.386+0000 7f0a2cf41640 1 --2- 172.21.15.37:0/3485125191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561617616000 0x561617518b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.386+0000 7f0a2c740640 1 -- 172.21.15.37:0/3485125191 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5616168db180 msgr2=0x561617518580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.386+0000 7f0a2c740640 1 --2- 172.21.15.37:0/3485125191 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5616168db180 0x561617518580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.386+0000 7f0a2c740640 1 -- 172.21.15.37:0/3485125191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561617616000 msgr2=0x561617518b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.386+0000 7f0a2c740640 1 --2- 172.21.15.37:0/3485125191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561617616000 0x561617518b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.386+0000 7f0a2c740640 1 -- 172.21.15.37:0/3485125191 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561616784780 con 0x5616168dba80 2024-09-17T04:34:23.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.386+0000 7f0a2c740640 1 --2- 172.21.15.37:0/3485125191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616168dba80 0x561617518000 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x5616175757d0 tx=0x561616845020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7a97d3c3237739 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.386+0000 7f0a29f3b640 1 -- 172.21.15.37:0/3485125191 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56161273c340 con 0x5616168dba80 2024-09-17T04:34:23.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.386+0000 7f0a29f3b640 1 -- 172.21.15.37:0/3485125191 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x561612743180 con 0x5616168dba80 2024-09-17T04:34:23.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.386+0000 7f0a29f3b640 1 -- 172.21.15.37:0/3485125191 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56161273cea0 con 0x5616168dba80 2024-09-17T04:34:23.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.386+0000 7f0a2df99280 1 -- 172.21.15.37:0/3485125191 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616167854a0 con 0x5616168dba80 2024-09-17T04:34:23.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.386+0000 7f0a2df99280 1 -- 172.21.15.37:0/3485125191 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) -- 0x561616784000 con 0x5616168dba80 2024-09-17T04:34:23.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.386+0000 7f0a2df99280 4 waiting for MDS map... 2024-09-17T04:34:23.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.387+0000 7f0a29f3b640 1 -- 172.21.15.37:0/3485125191 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x561617630600 con 0x5616168dba80 2024-09-17T04:34:23.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.387+0000 7f0a29f3b640 1 -- 172.21.15.37:0/3485125191 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 4143+0+0 (secure 0 0 0) 0x5616167a1180 con 0x5616168dba80 2024-09-17T04:34:23.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.387+0000 7f0a2df99280 4 Got MDS map 22 2024-09-17T04:34:23.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.387+0000 7f0a2df99280 10 main: main 2024-09-17T04:34:23.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.387+0000 7f0a2df99280 4 main: connecting to RADOS... 2024-09-17T04:34:23.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.388+0000 7f0a2df99280 1 Processor -- start 2024-09-17T04:34:23.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.388+0000 7f0a2df99280 1 -- start start 2024-09-17T04:34:23.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.389+0000 7f0a2df99280 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x561617616900 0x561617519080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.389+0000 7f0a2df99280 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561617616d80 0x561617519600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.389+0000 7f0a2df99280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561617617200 0x561617519b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.389+0000 7f0a2df99280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561617532600 con 0x561617617200 2024-09-17T04:34:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.389+0000 7f0a2df99280 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561617532780 con 0x561617616d80 2024-09-17T04:34:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.389+0000 7f0a2df99280 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x561617532900 con 0x561617616900 2024-09-17T04:34:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.389+0000 7f0a2bf3f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561617617200 0x561617519b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.389+0000 7f0a2bf3f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561617617200 0x561617519b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38676/0 (socket says 172.21.15.37:38676) 2024-09-17T04:34:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.389+0000 7f0a2bf3f640 1 -- 172.21.15.37:0/1125116452 learned_addr learned my addr 172.21.15.37:0/1125116452 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.389+0000 7f0a2cf41640 1 --2- 172.21.15.37:0/1125116452 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x561617616900 0x561617519080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.389+0000 7f0a2c740640 1 --2- 172.21.15.37:0/1125116452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561617616d80 0x561617519600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.389+0000 7f0a2bf3f640 1 -- 172.21.15.37:0/1125116452 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x561617616900 msgr2=0x561617519080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.389+0000 7f0a2bf3f640 1 --2- 172.21.15.37:0/1125116452 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x561617616900 0x561617519080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.389+0000 7f0a2bf3f640 1 -- 172.21.15.37:0/1125116452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561617616d80 msgr2=0x561617519600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.389+0000 7f0a2bf3f640 1 --2- 172.21.15.37:0/1125116452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561617616d80 0x561617519600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.389+0000 7f0a2bf3f640 1 -- 172.21.15.37:0/1125116452 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616167843c0 con 0x561617617200 2024-09-17T04:34:23.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.390+0000 7f0a2bf3f640 1 --2- 172.21.15.37:0/1125116452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561617617200 0x561617519b80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x56161761cc90 tx=0x56161768c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e18977ec8a2283c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.390+0000 7f0a27f37640 1 -- 172.21.15.37:0/1125116452 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56161273c340 con 0x561617617200 2024-09-17T04:34:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.390+0000 7f0a27f37640 1 -- 172.21.15.37:0/1125116452 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x561612742380 con 0x561617617200 2024-09-17T04:34:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.390+0000 7f0a27f37640 1 -- 172.21.15.37:0/1125116452 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56161273cea0 con 0x561617617200 2024-09-17T04:34:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.390+0000 7f0a2df99280 1 -- 172.21.15.37:0/1125116452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561617617200 msgr2=0x561617519b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.390+0000 7f0a2df99280 1 --2- 172.21.15.37:0/1125116452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561617617200 0x561617519b80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x56161761cc90 tx=0x56161768c5a0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.390+0000 7f0a2df99280 1 -- 172.21.15.37:0/1125116452 shutdown_connections 2024-09-17T04:34:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.390+0000 7f0a2df99280 1 --2- 172.21.15.37:0/1125116452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561617617200 0x561617519b80 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.390+0000 7f0a2df99280 1 --2- 172.21.15.37:0/1125116452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561617616d80 0x561617519600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.390+0000 7f0a2df99280 1 --2- 172.21.15.37:0/1125116452 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x561617616900 0x561617519080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.390+0000 7f0a2df99280 1 -- 172.21.15.37:0/1125116452 >> 172.21.15.37:0/1125116452 conn(0x561617616480 msgr2=0x561616861800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.390+0000 7f0a2df99280 1 -- 172.21.15.37:0/1125116452 shutdown_connections 2024-09-17T04:34:23.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.390+0000 7f0a2df99280 1 -- 172.21.15.37:0/1125116452 wait complete. 2024-09-17T04:34:23.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.391+0000 7f0a2df99280 1 Processor -- start 2024-09-17T04:34:23.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.391+0000 7f0a2df99280 1 -- start start 2024-09-17T04:34:23.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.392+0000 7f0a2df99280 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x561617616900 0x561617519080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.392+0000 7f0a2df99280 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561617616d80 0x561617519600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.392+0000 7f0a2df99280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561617617200 0x561617519b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.392+0000 7f0a2df99280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561617533980 con 0x561617617200 2024-09-17T04:34:23.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.392+0000 7f0a2df99280 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561617532300 con 0x561617616d80 2024-09-17T04:34:23.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.392+0000 7f0a2df99280 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x561617532c00 con 0x561617616900 2024-09-17T04:34:23.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.392+0000 7f0a2bf3f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561617617200 0x561617519b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.392+0000 7f0a2bf3f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561617617200 0x561617519b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38688/0 (socket says 172.21.15.37:38688) 2024-09-17T04:34:23.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.393+0000 7f0a2bf3f640 1 -- 172.21.15.37:0/3574141684 learned_addr learned my addr 172.21.15.37:0/3574141684 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.393+0000 7f0a2c740640 1 --2- 172.21.15.37:0/3574141684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561617616d80 0x561617519600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.393+0000 7f0a2cf41640 1 --2- 172.21.15.37:0/3574141684 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x561617616900 0x561617519080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.393+0000 7f0a2bf3f640 1 -- 172.21.15.37:0/3574141684 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x561617616900 msgr2=0x561617519080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.393+0000 7f0a2bf3f640 1 --2- 172.21.15.37:0/3574141684 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x561617616900 0x561617519080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.393+0000 7f0a2bf3f640 1 -- 172.21.15.37:0/3574141684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561617616d80 msgr2=0x561617519600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.393+0000 7f0a2bf3f640 1 --2- 172.21.15.37:0/3574141684 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561617616d80 0x561617519600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.393+0000 7f0a2bf3f640 1 -- 172.21.15.37:0/3574141684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561616784960 con 0x561617617200 2024-09-17T04:34:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.393+0000 7f0a2bf3f640 1 --2- 172.21.15.37:0/3574141684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561617617200 0x561617519b80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x56161761d950 tx=0x56161768c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.393+0000 7f0a26734640 1 -- 172.21.15.37:0/3574141684 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56161273d040 con 0x561617617200 2024-09-17T04:34:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.393+0000 7f0a26734640 1 -- 172.21.15.37:0/3574141684 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x561612742e00 con 0x561617617200 2024-09-17T04:34:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.393+0000 7f0a26734640 1 -- 172.21.15.37:0/3574141684 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56161273c340 con 0x561617617200 2024-09-17T04:34:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.393+0000 7f0a2df99280 1 -- 172.21.15.37:0/3574141684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561616785680 con 0x561617617200 2024-09-17T04:34:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.393+0000 7f0a2df99280 1 -- 172.21.15.37:0/3574141684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561616785860 con 0x561617617200 2024-09-17T04:34:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.393+0000 7f0a2df99280 4 main: resolving pool 6 2024-09-17T04:34:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.395+0000 7f0a26734640 1 -- 172.21.15.37:0/3574141684 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x56161753a900 con 0x561617617200 2024-09-17T04:34:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.395+0000 7f0a26734640 1 --2- 172.21.15.37:0/3574141684 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x561617619600 0x56161751a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.395+0000 7f0a26734640 1 -- 172.21.15.37:0/3574141684 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x56161767f400 con 0x561617617200 2024-09-17T04:34:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.395+0000 7f0a2df99280 4 main: creating IoCtx.. 2024-09-17T04:34:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.395+0000 7f0a2df99280 1 --2- 172.21.15.37:0/3574141684 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x561617617680 0x56161751a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.395+0000 7f0a2df99280 1 -- 172.21.15.37:0/3574141684 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:1 6.13 6:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x561617698800 con 0x561617617680 2024-09-17T04:34:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.395+0000 7f0a2cf41640 1 --2- 172.21.15.37:0/3574141684 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x561617619600 0x56161751a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.395+0000 7f0a2c740640 1 --2- 172.21.15.37:0/3574141684 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x561617617680 0x56161751a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.396+0000 7f0a2c740640 1 --2- 172.21.15.37:0/3574141684 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x561617617680 0x56161751a680 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.396+0000 7f0a2cf41640 1 --2- 172.21.15.37:0/3574141684 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x561617619600 0x56161751a100 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x56161680ccf0 tx=0x56161768ccc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.397+0000 7f0a2c740640 1 -- 172.21.15.37:0/3574141684 <== osd.3 v2:172.21.15.37:6800/809975826 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v41'2 uv2 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x56161683e000 con 0x561617617680 2024-09-17T04:34:23.405 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T04:34:23.406 INFO:teuthology.orchestra.run.smithi037.stdout: "0": { 2024-09-17T04:34:23.406 INFO:teuthology.orchestra.run.smithi037.stdout: "data": {}, 2024-09-17T04:34:23.406 INFO:teuthology.orchestra.run.smithi037.stdout: "result": 0 2024-09-17T04:34:23.406 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:34:23.406 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T04:34:23.406 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.397+0000 7f0a2df99280 1 -- 172.21.15.37:0/3574141684 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x561617617680 msgr2=0x56161751a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.397+0000 7f0a2df99280 1 --2- 172.21.15.37:0/3574141684 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x561617617680 0x56161751a680 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.397+0000 7f0a2df99280 1 -- 172.21.15.37:0/3574141684 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x561617619600 msgr2=0x56161751a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.397+0000 7f0a2df99280 1 --2- 172.21.15.37:0/3574141684 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x561617619600 0x56161751a100 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x56161680ccf0 tx=0x56161768ccc0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.397+0000 7f0a2df99280 1 -- 172.21.15.37:0/3574141684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561617617200 msgr2=0x561617519b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.397+0000 7f0a2df99280 1 --2- 172.21.15.37:0/3574141684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561617617200 0x561617519b80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x56161761d950 tx=0x56161768c780 comp rx=0 tx=0).stop 2024-09-17T04:34:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.397+0000 7f0a2bf3f640 1 -- 172.21.15.37:0/3574141684 reap_dead start 2024-09-17T04:34:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.397+0000 7f0a2df99280 1 -- 172.21.15.37:0/3574141684 shutdown_connections 2024-09-17T04:34:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.397+0000 7f0a2df99280 1 -- 172.21.15.37:0/3574141684 >> 172.21.15.37:0/3574141684 conn(0x561617616480 msgr2=0x561616861800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.398+0000 7f0a2df99280 1 -- 172.21.15.37:0/3574141684 shutdown_connections 2024-09-17T04:34:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.398+0000 7f0a2df99280 1 -- 172.21.15.37:0/3574141684 wait complete. 2024-09-17T04:34:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.398+0000 7f0a2df99280 1 -- 172.21.15.37:0/3485125191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616168dba80 msgr2=0x561617518000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.398+0000 7f0a2df99280 1 --2- 172.21.15.37:0/3485125191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616168dba80 0x561617518000 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x5616175757d0 tx=0x561616845020 comp rx=0 tx=0).stop 2024-09-17T04:34:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.398+0000 7f0a2df99280 1 -- 172.21.15.37:0/3485125191 shutdown_connections 2024-09-17T04:34:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.398+0000 7f0a2df99280 1 --2- 172.21.15.37:0/3485125191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561617616000 0x561617518b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.398+0000 7f0a2df99280 1 --2- 172.21.15.37:0/3485125191 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5616168db180 0x561617518580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.398+0000 7f0a2df99280 1 --2- 172.21.15.37:0/3485125191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616168dba80 0x561617518000 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.398+0000 7f0a2df99280 1 -- 172.21.15.37:0/3485125191 >> 172.21.15.37:0/3485125191 conn(0x5616168db600 msgr2=0x561616861000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.398+0000 7f0a2df99280 1 -- 172.21.15.37:0/3485125191 shutdown_connections 2024-09-17T04:34:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.398+0000 7f0a2df99280 1 -- 172.21.15.37:0/3485125191 wait complete. 2024-09-17T04:34:23.410 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'recovery_fs:0', 'reset', 'inode'] in time 0:00:00.077590, result: { "0": { "data": {}, "result": 0 } } 2024-09-17T04:34:23.410 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --alternate-pool recovery_fs_metadata --filesystem cephfs cephfs_data 2024-09-17T04:34:23.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.467+0000 7fb35801c740 1 -- start start 2024-09-17T04:34:23.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.467+0000 7fb35801c740 1 Processor -- start 2024-09-17T04:34:23.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.468+0000 7fb35801c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555b27e3680 0x5555b3410580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.468+0000 7fb35801c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555b27e3b00 0x5555b3410000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.468+0000 7fb35801c740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5555b27e3f80 0x5555b3410b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.468+0000 7fb35801c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5555b342a180 con 0x5555b27e3680 2024-09-17T04:34:23.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.468+0000 7fb35801c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5555b342a300 con 0x5555b27e3b00 2024-09-17T04:34:23.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.468+0000 7fb35801c740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x5555b342a480 con 0x5555b27e3f80 2024-09-17T04:34:23.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.468+0000 7fb355d31640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555b27e3680 0x5555b3410580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.468+0000 7fb355d31640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555b27e3680 0x5555b3410580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38716/0 (socket says 172.21.15.37:38716) 2024-09-17T04:34:23.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.468+0000 7fb355d31640 1 -- 172.21.15.37:0/2319491477 learned_addr learned my addr 172.21.15.37:0/2319491477 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.468+0000 7fb356532640 1 --2- 172.21.15.37:0/2319491477 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5555b27e3f80 0x5555b3410b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.468+0000 7fb355530640 1 --2- 172.21.15.37:0/2319491477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555b27e3b00 0x5555b3410000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.469+0000 7fb355d31640 1 -- 172.21.15.37:0/2319491477 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5555b27e3f80 msgr2=0x5555b3410b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.469+0000 7fb355d31640 1 --2- 172.21.15.37:0/2319491477 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5555b27e3f80 0x5555b3410b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.469+0000 7fb355d31640 1 -- 172.21.15.37:0/2319491477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555b27e3b00 msgr2=0x5555b3410000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.469+0000 7fb355d31640 1 --2- 172.21.15.37:0/2319491477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555b27e3b00 0x5555b3410000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.469+0000 7fb355d31640 1 -- 172.21.15.37:0/2319491477 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555ae637680 con 0x5555b27e3680 2024-09-17T04:34:23.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.469+0000 7fb355d31640 1 --2- 172.21.15.37:0/2319491477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555b27e3680 0x5555b3410580 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x5555b350e0c0 tx=0x5555b273e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae9a9731944e966 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.469+0000 7fb35352c640 1 -- 172.21.15.37:0/2319491477 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5555ae630ea0 con 0x5555b27e3680 2024-09-17T04:34:23.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.469+0000 7fb35352c640 1 -- 172.21.15.37:0/2319491477 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5555b2690fc0 con 0x5555b27e3680 2024-09-17T04:34:23.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.469+0000 7fb35352c640 1 -- 172.21.15.37:0/2319491477 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5555ae631380 con 0x5555b27e3680 2024-09-17T04:34:23.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.469+0000 7fb35801c740 1 -- 172.21.15.37:0/2319491477 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555ae637860 con 0x5555b27e3680 2024-09-17T04:34:23.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.469+0000 7fb35801c740 1 -- 172.21.15.37:0/2319491477 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5555ae6372c0 con 0x5555b27e3680 2024-09-17T04:34:23.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.469+0000 7fb35801c740 4 waiting for MDS map... 2024-09-17T04:34:23.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.469+0000 7fb35352c640 1 -- 172.21.15.37:0/2319491477 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x5555b3524600 con 0x5555b27e3680 2024-09-17T04:34:23.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.469+0000 7fb35352c640 1 -- 172.21.15.37:0/2319491477 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 4143+0+0 (secure 0 0 0) 0x5555b2695180 con 0x5555b27e3680 2024-09-17T04:34:23.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.469+0000 7fb35801c740 4 Got MDS map 22 2024-09-17T04:34:23.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.469+0000 7fb35801c740 4 datascan.main: Using metadata pool output 2024-09-17T04:34:23.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.469+0000 7fb35801c740 4 datascan.main: connecting to RADOS... 2024-09-17T04:34:23.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.470+0000 7fb35801c740 1 Processor -- start 2024-09-17T04:34:23.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.470+0000 7fb35801c740 1 -- start start 2024-09-17T04:34:23.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.471+0000 7fb35801c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555b27e2d80 0x5555b3411080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.471+0000 7fb35801c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555b27e2900 0x5555b3411600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.471+0000 7fb35801c740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5555b27e4880 0x5555b3411b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.471+0000 7fb35801c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5555b342a600 con 0x5555b27e2900 2024-09-17T04:34:23.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.471+0000 7fb35801c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5555b342a780 con 0x5555b27e2d80 2024-09-17T04:34:23.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.471+0000 7fb35801c740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x5555b342a900 con 0x5555b27e4880 2024-09-17T04:34:23.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.471+0000 7fb355d31640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555b27e2900 0x5555b3411600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.471+0000 7fb355d31640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555b27e2900 0x5555b3411600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38726/0 (socket says 172.21.15.37:38726) 2024-09-17T04:34:23.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.471+0000 7fb355d31640 1 -- 172.21.15.37:0/1016057560 learned_addr learned my addr 172.21.15.37:0/1016057560 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.472+0000 7fb355530640 1 --2- 172.21.15.37:0/1016057560 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5555b27e4880 0x5555b3411b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.472+0000 7fb356532640 1 --2- 172.21.15.37:0/1016057560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555b27e2d80 0x5555b3411080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.472+0000 7fb355d31640 1 -- 172.21.15.37:0/1016057560 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5555b27e4880 msgr2=0x5555b3411b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.472+0000 7fb355d31640 1 --2- 172.21.15.37:0/1016057560 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5555b27e4880 0x5555b3411b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.472+0000 7fb355d31640 1 -- 172.21.15.37:0/1016057560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555b27e2d80 msgr2=0x5555b3411080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.472+0000 7fb355d31640 1 --2- 172.21.15.37:0/1016057560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555b27e2d80 0x5555b3411080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.472+0000 7fb355d31640 1 -- 172.21.15.37:0/1016057560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555ae6374a0 con 0x5555b27e2900 2024-09-17T04:34:23.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.472+0000 7fb356532640 1 --2- 172.21.15.37:0/1016057560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555b27e2d80 0x5555b3411080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:23.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.472+0000 7fb355d31640 1 --2- 172.21.15.37:0/1016057560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555b27e2900 0x5555b3411600 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x5555b350ea50 tx=0x5555b2716540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca3fd6b5215f4aa server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.472+0000 7fb351528640 1 -- 172.21.15.37:0/1016057560 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5555ae6311e0 con 0x5555b27e2900 2024-09-17T04:34:23.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.472+0000 7fb351528640 1 -- 172.21.15.37:0/1016057560 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5555b2690e00 con 0x5555b27e2900 2024-09-17T04:34:23.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.472+0000 7fb351528640 1 -- 172.21.15.37:0/1016057560 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5555ae630ea0 con 0x5555b27e2900 2024-09-17T04:34:23.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.473+0000 7fb35801c740 1 -- 172.21.15.37:0/1016057560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555b27e2900 msgr2=0x5555b3411600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.473+0000 7fb35801c740 1 --2- 172.21.15.37:0/1016057560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555b27e2900 0x5555b3411600 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x5555b350ea50 tx=0x5555b2716540 comp rx=0 tx=0).stop 2024-09-17T04:34:23.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.473+0000 7fb35801c740 1 -- 172.21.15.37:0/1016057560 shutdown_connections 2024-09-17T04:34:23.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.473+0000 7fb35801c740 1 --2- 172.21.15.37:0/1016057560 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5555b27e4880 0x5555b3411b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.473+0000 7fb35801c740 1 --2- 172.21.15.37:0/1016057560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555b27e2900 0x5555b3411600 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.473+0000 7fb35801c740 1 --2- 172.21.15.37:0/1016057560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555b27e2d80 0x5555b3411080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.473+0000 7fb35801c740 1 -- 172.21.15.37:0/1016057560 >> 172.21.15.37:0/1016057560 conn(0x5555b27e3200 msgr2=0x5555b275d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.473+0000 7fb35801c740 1 -- 172.21.15.37:0/1016057560 shutdown_connections 2024-09-17T04:34:23.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.473+0000 7fb35801c740 1 -- 172.21.15.37:0/1016057560 wait complete. 2024-09-17T04:34:23.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.474+0000 7fb35801c740 1 Processor -- start 2024-09-17T04:34:23.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.474+0000 7fb35801c740 1 -- start start 2024-09-17T04:34:23.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.475+0000 7fb35801c740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5555b27e2900 0x5555b3411600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.475+0000 7fb35801c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555b27e2d80 0x5555b3411080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.475+0000 7fb35801c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555b27e4880 0x5555b3411b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.475+0000 7fb35801c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5555b342bc80 con 0x5555b27e2d80 2024-09-17T04:34:23.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.475+0000 7fb35801c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5555b342aa80 con 0x5555b27e4880 2024-09-17T04:34:23.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.475+0000 7fb35801c740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x5555b342bb00 con 0x5555b27e2900 2024-09-17T04:34:23.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.475+0000 7fb355d31640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555b27e2d80 0x5555b3411080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.475+0000 7fb355d31640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555b27e2d80 0x5555b3411080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38736/0 (socket says 172.21.15.37:38736) 2024-09-17T04:34:23.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.475+0000 7fb355d31640 1 -- 172.21.15.37:0/2608060258 learned_addr learned my addr 172.21.15.37:0/2608060258 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.475+0000 7fb356532640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5555b27e2900 0x5555b3411600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.475+0000 7fb355530640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555b27e4880 0x5555b3411b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.475+0000 7fb355d31640 1 -- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5555b27e2900 msgr2=0x5555b3411600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.475+0000 7fb355d31640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5555b27e2900 0x5555b3411600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.475+0000 7fb355d31640 1 -- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555b27e4880 msgr2=0x5555b3411b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.475+0000 7fb355d31640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555b27e4880 0x5555b3411b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.475+0000 7fb355d31640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555ae637680 con 0x5555b27e2d80 2024-09-17T04:34:23.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.475+0000 7fb355530640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555b27e4880 0x5555b3411b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:23.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.475+0000 7fb355d31640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555b27e2d80 0x5555b3411080 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x5555b350f860 tx=0x5555b3542cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.475+0000 7fb34fd25640 1 -- 172.21.15.37:0/2608060258 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5555ae631380 con 0x5555b27e2d80 2024-09-17T04:34:23.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.475+0000 7fb34fd25640 1 -- 172.21.15.37:0/2608060258 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5555b2690c40 con 0x5555b27e2d80 2024-09-17T04:34:23.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.475+0000 7fb35801c740 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5555ae637860 con 0x5555b27e2d80 2024-09-17T04:34:23.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.476+0000 7fb34fd25640 1 -- 172.21.15.37:0/2608060258 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5555ae631040 con 0x5555b27e2d80 2024-09-17T04:34:23.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.476+0000 7fb35801c740 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555ae6370e0 con 0x5555b27e2d80 2024-09-17T04:34:23.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.476+0000 7fb35801c740 4 datascan.init: forcing metadata pool 'recovery_fs_metadata' 2024-09-17T04:34:23.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.477+0000 7fb34fd25640 1 -- 172.21.15.37:0/2608060258 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x5555b3582900 con 0x5555b27e2d80 2024-09-17T04:34:23.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.477+0000 7fb34fd25640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x5555b27e5a80 0x5555b3412100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.477+0000 7fb34fd25640 1 -- 172.21.15.37:0/2608060258 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x5555b35a0600 con 0x5555b27e2d80 2024-09-17T04:34:23.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.477+0000 7fb35801c740 4 datascan.main: data pool 'cephfs_data' has ID 5 2024-09-17T04:34:23.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.477+0000 7fb35801c740 4 datascan.main: opening data pool 'cephfs_data' 2024-09-17T04:34:23.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.478+0000 7fb35801c740 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x5555b27e4d00 0x5555b3412680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.478+0000 7fb35801c740 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:1 5.0 5:00000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3599000 con 0x5555b27e4d00 2024-09-17T04:34:23.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.478+0000 7fb356532640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x5555b27e5a80 0x5555b3412100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.478+0000 7fb355530640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x5555b27e4d00 0x5555b3412680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.478+0000 7fb355530640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x5555b27e4d00 0x5555b3412680 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.478+0000 7fb356532640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x5555b27e5a80 0x5555b3412100 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x5555b272f5c0 tx=0x5555b35430e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.478+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.3 v2:172.21.15.37:6800/809975826 1 ==== osd_op_reply(1 [pgnls start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a000 con 0x5555b27e4d00 2024-09-17T04:34:23.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.478+0000 7fb350d27640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x5555b3442000 0x5555b2a58000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.479+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:2 5.20 5:04000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3446000 con 0x5555b3442000 2024-09-17T04:34:23.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.479+0000 7fb356532640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x5555b3442000 0x5555b2a58000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.479+0000 7fb356532640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x5555b3442000 0x5555b2a58000 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.479+0000 7fb356532640 1 -- 172.21.15.37:0/2608060258 <== osd.7 v2:172.21.15.92:6802/3444787223 1 ==== osd_op_reply(2 [pgnls start_epoch 41 out=49b] v33'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a500 con 0x5555b3442000 2024-09-17T04:34:23.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.480+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:3 5.10 5:08000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3446400 con 0x5555b3442000 2024-09-17T04:34:23.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.480+0000 7fb356532640 1 -- 172.21.15.37:0/2608060258 <== osd.7 v2:172.21.15.92:6802/3444787223 2 ==== osd_op_reply(3 [pgnls start_epoch 41 out=49b] v33'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a500 con 0x5555b3442000 2024-09-17T04:34:23.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.480+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:4 5.30 5:0c000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3446800 con 0x5555b3442000 2024-09-17T04:34:23.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.480+0000 7fb356532640 1 -- 172.21.15.37:0/2608060258 <== osd.7 v2:172.21.15.92:6802/3444787223 3 ==== osd_op_reply(4 [pgnls start_epoch 41 out=49b] v33'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a500 con 0x5555b3442000 2024-09-17T04:34:23.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.480+0000 7fb351528640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x5555b3442480 0x5555b2a58580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.480+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:5 5.8 5:10000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3447000 con 0x5555b3442480 2024-09-17T04:34:23.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.480+0000 7fb355d31640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x5555b3442480 0x5555b2a58580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.481+0000 7fb355d31640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x5555b3442480 0x5555b2a58580 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.481+0000 7fb355d31640 1 -- 172.21.15.37:0/2608060258 <== osd.2 v2:172.21.15.37:6816/579720176 1 ==== osd_op_reply(5 [pgnls start_epoch 41 out=49b] v33'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a780 con 0x5555b3442480 2024-09-17T04:34:23.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.481+0000 7fb350d27640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x5555b3442900 0x5555b2a58b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.481+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:6 5.28 5:14000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3446c00 con 0x5555b3442900 2024-09-17T04:34:23.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.481+0000 7fb355530640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x5555b3442900 0x5555b2a58b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.482+0000 7fb355530640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x5555b3442900 0x5555b2a58b00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.482+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.5 v2:172.21.15.92:6826/248263832 1 ==== osd_op_reply(6 [pgnls start_epoch 41 out=49b] v33'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a000 con 0x5555b3442900 2024-09-17T04:34:23.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.482+0000 7fb351528640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x5555b3443200 0x5555b2a59600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.482+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:7 5.18 5:18000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3447400 con 0x5555b3443200 2024-09-17T04:34:23.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.482+0000 7fb356532640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x5555b3443200 0x5555b2a59600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.483+0000 7fb356532640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x5555b3443200 0x5555b2a59600 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.483+0000 7fb356532640 1 -- 172.21.15.37:0/2608060258 <== osd.4 v2:172.21.15.92:6818/2509785752 1 ==== osd_op_reply(7 [pgnls start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a500 con 0x5555b3443200 2024-09-17T04:34:23.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.483+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:8 5.38 5:1c000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3446000 con 0x5555b3442000 2024-09-17T04:34:23.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.484+0000 7fb356532640 1 -- 172.21.15.37:0/2608060258 <== osd.7 v2:172.21.15.92:6802/3444787223 4 ==== osd_op_reply(8 [pgnls start_epoch 41 out=49b] v33'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a500 con 0x5555b3442000 2024-09-17T04:34:23.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.484+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:9 5.4 5:20000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3447400 con 0x5555b3442000 2024-09-17T04:34:23.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.484+0000 7fb356532640 1 -- 172.21.15.37:0/2608060258 <== osd.7 v2:172.21.15.92:6802/3444787223 5 ==== osd_op_reply(9 [pgnls start_epoch 41 out=49b] v33'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a500 con 0x5555b3442000 2024-09-17T04:34:23.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.484+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:10 5.24 5:24000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3446400 con 0x5555b3442900 2024-09-17T04:34:23.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.484+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.5 v2:172.21.15.92:6826/248263832 2 ==== osd_op_reply(10 [pgnls start_epoch 41 out=49b] v33'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a000 con 0x5555b3442900 2024-09-17T04:34:23.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.484+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3447800 con 0x5555b27e4d00 2024-09-17T04:34:23.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.485+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.3 v2:172.21.15.37:6800/809975826 2 ==== osd_op_reply(11 [pgnls start_epoch 41 out=49b] v33'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a000 con 0x5555b27e4d00 2024-09-17T04:34:23.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.485+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:12 5.34 5:2c000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3446800 con 0x5555b3442900 2024-09-17T04:34:23.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.485+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.5 v2:172.21.15.92:6826/248263832 3 ==== osd_op_reply(12 [pgnls start_epoch 41 out=49b] v33'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a000 con 0x5555b3442900 2024-09-17T04:34:23.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.485+0000 7fb351528640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x5555b3443680 0x5555b2a59b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.485+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:13 5.c 5:30000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3447c00 con 0x5555b3443680 2024-09-17T04:34:23.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.485+0000 7fb355d31640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x5555b3443680 0x5555b2a59b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.486+0000 7fb355d31640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x5555b3443680 0x5555b2a59b80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.486+0000 7fb355d31640 1 -- 172.21.15.37:0/2608060258 <== osd.1 v2:172.21.15.37:6824/2657955935 1 ==== osd_op_reply(13 [pgnls start_epoch 41 out=49b] v33'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a780 con 0x5555b3443680 2024-09-17T04:34:23.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.486+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:14 5.2c 5:34000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3446c00 con 0x5555b3442480 2024-09-17T04:34:23.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.486+0000 7fb355d31640 1 -- 172.21.15.37:0/2608060258 <== osd.2 v2:172.21.15.37:6816/579720176 2 ==== osd_op_reply(14 [pgnls start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a780 con 0x5555b3442480 2024-09-17T04:34:23.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.486+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:15 5.1c 5:38000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3446400 con 0x5555b3443200 2024-09-17T04:34:23.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.486+0000 7fb356532640 1 -- 172.21.15.37:0/2608060258 <== osd.4 v2:172.21.15.92:6818/2509785752 2 ==== osd_op_reply(15 [pgnls start_epoch 41 out=49b] v33'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a500 con 0x5555b3443200 2024-09-17T04:34:23.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.486+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:16 5.3c 5:3c000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3599000 con 0x5555b3443680 2024-09-17T04:34:23.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.487+0000 7fb355d31640 1 -- 172.21.15.37:0/2608060258 <== osd.1 v2:172.21.15.37:6824/2657955935 2 ==== osd_op_reply(16 [pgnls start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a780 con 0x5555b3443680 2024-09-17T04:34:23.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.487+0000 7fb351528640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] conn(0x5555b3443b00 0x5555b2a5a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.487+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] -- osd_op(unknown.0.0:17 5.2 5:40000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3598c00 con 0x5555b3443b00 2024-09-17T04:34:23.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.487+0000 7fb355530640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] conn(0x5555b3443b00 0x5555b2a5a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.487+0000 7fb355530640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] conn(0x5555b3443b00 0x5555b2a5a100 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.488+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.6 v2:172.21.15.92:6810/4078365985 1 ==== osd_op_reply(17 [pgnls start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a000 con 0x5555b3443b00 2024-09-17T04:34:23.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.488+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:18 5.22 5:44000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555ae60b800 con 0x5555b3442000 2024-09-17T04:34:23.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.488+0000 7fb356532640 1 -- 172.21.15.37:0/2608060258 <== osd.7 v2:172.21.15.92:6802/3444787223 6 ==== osd_op_reply(18 [pgnls start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a500 con 0x5555b3442000 2024-09-17T04:34:23.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.488+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:19 5.12 5:48000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3598800 con 0x5555b3443680 2024-09-17T04:34:23.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.488+0000 7fb355d31640 1 -- 172.21.15.37:0/2608060258 <== osd.1 v2:172.21.15.37:6824/2657955935 3 ==== osd_op_reply(19 [pgnls start_epoch 41 out=49b] v33'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a780 con 0x5555b3443680 2024-09-17T04:34:23.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.488+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:20 5.32 5:4c000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555ae60bc00 con 0x5555b27e4d00 2024-09-17T04:34:23.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.489+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.3 v2:172.21.15.37:6800/809975826 3 ==== osd_op_reply(20 [pgnls start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a000 con 0x5555b27e4d00 2024-09-17T04:34:23.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.489+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:21 5.a 5:50000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3447000 con 0x5555b3442480 2024-09-17T04:34:23.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.489+0000 7fb355d31640 1 -- 172.21.15.37:0/2608060258 <== osd.2 v2:172.21.15.37:6816/579720176 3 ==== osd_op_reply(21 [pgnls start_epoch 41 out=49b] v33'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a780 con 0x5555b3442480 2024-09-17T04:34:23.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.489+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:22 5.2a 5:54000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3599000 con 0x5555b3443200 2024-09-17T04:34:23.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.489+0000 7fb356532640 1 -- 172.21.15.37:0/2608060258 <== osd.4 v2:172.21.15.92:6818/2509785752 3 ==== osd_op_reply(22 [pgnls start_epoch 41 out=49b] v33'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a500 con 0x5555b3443200 2024-09-17T04:34:23.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.489+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:23 5.1a 5:58000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3447c00 con 0x5555b3442000 2024-09-17T04:34:23.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.490+0000 7fb356532640 1 -- 172.21.15.37:0/2608060258 <== osd.7 v2:172.21.15.92:6802/3444787223 7 ==== osd_op_reply(23 [pgnls start_epoch 41 out=49b] v33'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a500 con 0x5555b3442000 2024-09-17T04:34:23.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.490+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:24 5.3a 5:5c000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3599000 con 0x5555b27e4d00 2024-09-17T04:34:23.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.490+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.3 v2:172.21.15.37:6800/809975826 4 ==== osd_op_reply(24 [pgnls start_epoch 41 out=49b] v33'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a000 con 0x5555b27e4d00 2024-09-17T04:34:23.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.490+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:25 5.6 5:60000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3446c00 con 0x5555b3442480 2024-09-17T04:34:23.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.490+0000 7fb355d31640 1 -- 172.21.15.37:0/2608060258 <== osd.2 v2:172.21.15.37:6816/579720176 4 ==== osd_op_reply(25 [pgnls start_epoch 41 out=49b] v33'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a780 con 0x5555b3442480 2024-09-17T04:34:23.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.490+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:26 5.26 5:64000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555ae60b800 con 0x5555b27e4d00 2024-09-17T04:34:23.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.491+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.3 v2:172.21.15.37:6800/809975826 5 ==== osd_op_reply(26 [pgnls start_epoch 41 out=49b] v33'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a000 con 0x5555b27e4d00 2024-09-17T04:34:23.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.491+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:27 5.16 5:68000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555ae60bc00 con 0x5555b3442900 2024-09-17T04:34:23.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.491+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.5 v2:172.21.15.92:6826/248263832 4 ==== osd_op_reply(27 [pgnls start_epoch 41 out=49b] v33'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a000 con 0x5555b3442900 2024-09-17T04:34:23.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.491+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:28 5.36 5:6c000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3446000 con 0x5555b3443680 2024-09-17T04:34:23.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.491+0000 7fb355d31640 1 -- 172.21.15.37:0/2608060258 <== osd.1 v2:172.21.15.37:6824/2657955935 4 ==== osd_op_reply(28 [pgnls start_epoch 41 out=49b] v33'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a780 con 0x5555b3443680 2024-09-17T04:34:23.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.491+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:29 5.e 5:70000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555ae60b800 con 0x5555b3443200 2024-09-17T04:34:23.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.492+0000 7fb356532640 1 -- 172.21.15.37:0/2608060258 <== osd.4 v2:172.21.15.92:6818/2509785752 4 ==== osd_op_reply(29 [pgnls start_epoch 41 out=49b] v33'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a500 con 0x5555b3443200 2024-09-17T04:34:23.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.492+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:30 5.2e 5:74000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3447400 con 0x5555b27e4d00 2024-09-17T04:34:23.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.492+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.3 v2:172.21.15.37:6800/809975826 6 ==== osd_op_reply(30 [pgnls start_epoch 41 out=49b] v33'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a000 con 0x5555b27e4d00 2024-09-17T04:34:23.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.492+0000 7fb351528640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x5555b3443f80 0x5555b2a5a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.492+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:31 5.1e 5:78000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3598c00 con 0x5555b3443f80 2024-09-17T04:34:23.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.492+0000 7fb356532640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x5555b3443f80 0x5555b2a5a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.492+0000 7fb356532640 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x5555b3443f80 0x5555b2a5a680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.493+0000 7fb356532640 1 -- 172.21.15.37:0/2608060258 <== osd.0 v2:172.21.15.37:6808/2283587593 1 ==== osd_op_reply(31 [pgnls start_epoch 41 out=49b] v33'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a500 con 0x5555b3443f80 2024-09-17T04:34:23.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.493+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:32 5.3e 5:7c000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3446400 con 0x5555b27e4d00 2024-09-17T04:34:23.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.493+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.3 v2:172.21.15.37:6800/809975826 7 ==== osd_op_reply(32 [pgnls start_epoch 41 out=49b] v33'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a000 con 0x5555b27e4d00 2024-09-17T04:34:23.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.493+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:33 5.1 5:80000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3599000 con 0x5555b3443200 2024-09-17T04:34:23.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.493+0000 7fb356532640 1 -- 172.21.15.37:0/2608060258 <== osd.4 v2:172.21.15.92:6818/2509785752 5 ==== osd_op_reply(33 [pgnls start_epoch 41 out=49b] v33'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a500 con 0x5555b3443200 2024-09-17T04:34:23.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.494+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] -- osd_op(unknown.0.0:34 5.21 5:84000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b2a82000 con 0x5555b3443b00 2024-09-17T04:34:23.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.494+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.6 v2:172.21.15.92:6810/4078365985 2 ==== osd_op_reply(34 [pgnls start_epoch 41 out=49b] v33'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a000 con 0x5555b3443b00 2024-09-17T04:34:23.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.494+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] -- osd_op(unknown.0.0:35 5.11 5:88000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3447800 con 0x5555b3443b00 2024-09-17T04:34:23.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.494+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.6 v2:172.21.15.92:6810/4078365985 3 ==== osd_op_reply(35 [pgnls start_epoch 41 out=49b] v33'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a000 con 0x5555b3443b00 2024-09-17T04:34:23.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.494+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:36 5.31 5:8c000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b2a82400 con 0x5555b3442900 2024-09-17T04:34:23.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.495+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.5 v2:172.21.15.92:6826/248263832 5 ==== osd_op_reply(36 [pgnls start_epoch 41 out=49b] v33'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a000 con 0x5555b3442900 2024-09-17T04:34:23.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.495+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:37 5.9 5:90000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3446800 con 0x5555b3442000 2024-09-17T04:34:23.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.495+0000 7fb356532640 1 -- 172.21.15.37:0/2608060258 <== osd.7 v2:172.21.15.92:6802/3444787223 8 ==== osd_op_reply(37 [pgnls start_epoch 41 out=49b] v33'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a500 con 0x5555b3442000 2024-09-17T04:34:23.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.495+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:38 5.29 5:94000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b2a82800 con 0x5555b3443f80 2024-09-17T04:34:23.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.496+0000 7fb356532640 1 -- 172.21.15.37:0/2608060258 <== osd.0 v2:172.21.15.37:6808/2283587593 2 ==== osd_op_reply(38 [pgnls start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a500 con 0x5555b3443f80 2024-09-17T04:34:23.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.496+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:39 5.19 5:98000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b2a82800 con 0x5555b3443680 2024-09-17T04:34:23.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.496+0000 7fb355d31640 1 -- 172.21.15.37:0/2608060258 <== osd.1 v2:172.21.15.37:6824/2657955935 5 ==== osd_op_reply(39 [pgnls start_epoch 41 out=49b] v33'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a780 con 0x5555b3443680 2024-09-17T04:34:23.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.496+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:40 5.39 5:9c000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b2a82c00 con 0x5555b3442000 2024-09-17T04:34:23.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.497+0000 7fb356532640 1 -- 172.21.15.37:0/2608060258 <== osd.7 v2:172.21.15.92:6802/3444787223 9 ==== osd_op_reply(40 [pgnls start_epoch 41 out=49b] v33'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a500 con 0x5555b3442000 2024-09-17T04:34:23.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.497+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:41 5.5 5:a0000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b2a83c00 con 0x5555b3443f80 2024-09-17T04:34:23.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.497+0000 7fb356532640 1 -- 172.21.15.37:0/2608060258 <== osd.0 v2:172.21.15.37:6808/2283587593 3 ==== osd_op_reply(41 [pgnls start_epoch 41 out=49b] v33'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a500 con 0x5555b3443f80 2024-09-17T04:34:23.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.497+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:42 5.25 5:a4000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b2a83000 con 0x5555b3443680 2024-09-17T04:34:23.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.497+0000 7fb355d31640 1 -- 172.21.15.37:0/2608060258 <== osd.1 v2:172.21.15.37:6824/2657955935 6 ==== osd_op_reply(42 [pgnls start_epoch 41 out=49b] v33'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a780 con 0x5555b3443680 2024-09-17T04:34:23.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.497+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:43 5.15 5:a8000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3447c00 con 0x5555b3442900 2024-09-17T04:34:23.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.498+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.5 v2:172.21.15.92:6826/248263832 6 ==== osd_op_reply(43 [pgnls start_epoch 41 out=49b] v33'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a000 con 0x5555b3442900 2024-09-17T04:34:23.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.498+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:44 5.35 5:ac000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b2a83400 con 0x5555b3442000 2024-09-17T04:34:23.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.498+0000 7fb356532640 1 -- 172.21.15.37:0/2608060258 <== osd.7 v2:172.21.15.92:6802/3444787223 10 ==== osd_op_reply(44 [pgnls start_epoch 41 out=49b] v33'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a500 con 0x5555b3442000 2024-09-17T04:34:23.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.498+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:45 5.d 5:b0000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3446800 con 0x5555b3442480 2024-09-17T04:34:23.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.498+0000 7fb355d31640 1 -- 172.21.15.37:0/2608060258 <== osd.2 v2:172.21.15.37:6816/579720176 5 ==== osd_op_reply(45 [pgnls start_epoch 41 out=49b] v33'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a780 con 0x5555b3442480 2024-09-17T04:34:23.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.498+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:46 5.2d 5:b4000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b2a83800 con 0x5555b27e4d00 2024-09-17T04:34:23.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.499+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.3 v2:172.21.15.37:6800/809975826 8 ==== osd_op_reply(46 [pgnls start_epoch 41 out=49b] v33'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a000 con 0x5555b27e4d00 2024-09-17T04:34:23.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.499+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:47 5.1d 5:b8000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3598c00 con 0x5555b3443680 2024-09-17T04:34:23.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.499+0000 7fb355d31640 1 -- 172.21.15.37:0/2608060258 <== osd.1 v2:172.21.15.37:6824/2657955935 7 ==== osd_op_reply(47 [pgnls start_epoch 41 out=49b] v33'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a780 con 0x5555b3443680 2024-09-17T04:34:23.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.499+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:48 5.3d 5:bc000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b2a82800 con 0x5555b3442900 2024-09-17T04:34:23.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.499+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.5 v2:172.21.15.92:6826/248263832 7 ==== osd_op_reply(48 [pgnls start_epoch 41 out=49b] v33'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a000 con 0x5555b3442900 2024-09-17T04:34:23.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.499+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:49 5.3 5:c0000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3599000 con 0x5555b3443f80 2024-09-17T04:34:23.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.500+0000 7fb356532640 1 -- 172.21.15.37:0/2608060258 <== osd.0 v2:172.21.15.37:6808/2283587593 4 ==== osd_op_reply(49 [pgnls start_epoch 41 out=49b] v33'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a500 con 0x5555b3443f80 2024-09-17T04:34:23.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.500+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:50 5.23 5:c4000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b2a83000 con 0x5555b3443200 2024-09-17T04:34:23.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.500+0000 7fb356532640 1 -- 172.21.15.37:0/2608060258 <== osd.4 v2:172.21.15.92:6818/2509785752 6 ==== osd_op_reply(50 [pgnls start_epoch 41 out=49b] v33'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a500 con 0x5555b3443200 2024-09-17T04:34:23.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.500+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:51 5.13 5:c8000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555ae60b800 con 0x5555b27e4d00 2024-09-17T04:34:23.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.500+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.3 v2:172.21.15.37:6800/809975826 9 ==== osd_op_reply(51 [pgnls start_epoch 41 out=49b] v33'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a000 con 0x5555b27e4d00 2024-09-17T04:34:23.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.500+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:52 5.33 5:cc000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b2a82000 con 0x5555b3442480 2024-09-17T04:34:23.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.501+0000 7fb355d31640 1 -- 172.21.15.37:0/2608060258 <== osd.2 v2:172.21.15.37:6816/579720176 6 ==== osd_op_reply(52 [pgnls start_epoch 41 out=49b] v33'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a780 con 0x5555b3442480 2024-09-17T04:34:23.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.501+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:53 5.b 5:d0000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3446000 con 0x5555b3442480 2024-09-17T04:34:23.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.501+0000 7fb355d31640 1 -- 172.21.15.37:0/2608060258 <== osd.2 v2:172.21.15.37:6816/579720176 7 ==== osd_op_reply(53 [pgnls start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a780 con 0x5555b3442480 2024-09-17T04:34:23.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.501+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:54 5.2b 5:d4000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b2a82400 con 0x5555b3443200 2024-09-17T04:34:23.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.501+0000 7fb356532640 1 -- 172.21.15.37:0/2608060258 <== osd.4 v2:172.21.15.92:6818/2509785752 7 ==== osd_op_reply(54 [pgnls start_epoch 41 out=49b] v33'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a500 con 0x5555b3443200 2024-09-17T04:34:23.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.501+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:55 5.1b 5:d8000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3446400 con 0x5555b3442900 2024-09-17T04:34:23.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.502+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.5 v2:172.21.15.92:6826/248263832 8 ==== osd_op_reply(55 [pgnls start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a000 con 0x5555b3442900 2024-09-17T04:34:23.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.502+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:56 5.3b 5:dc000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555ae60bc00 con 0x5555b3442480 2024-09-17T04:34:23.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.502+0000 7fb355d31640 1 -- 172.21.15.37:0/2608060258 <== osd.2 v2:172.21.15.37:6816/579720176 8 ==== osd_op_reply(56 [pgnls start_epoch 41 out=49b] v33'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a780 con 0x5555b3442480 2024-09-17T04:34:23.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.502+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:57 5.7 5:e0000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3446800 con 0x5555b3442900 2024-09-17T04:34:23.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.502+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.5 v2:172.21.15.92:6826/248263832 9 ==== osd_op_reply(57 [pgnls start_epoch 41 out=81b] v33'8 uv8 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5555b273a000 con 0x5555b3442900 2024-09-17T04:34:23.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.502+0000 7fb35801c740 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:58 5.7 5:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x5555b3599400 con 0x5555b3442900 2024-09-17T04:34:23.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.503+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.5 v2:172.21.15.92:6826/248263832 10 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5555b273a000 con 0x5555b3442900 2024-09-17T04:34:23.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.503+0000 7fb35801c740 10 datascan.operator(): handling object 1099511627777.0 2024-09-17T04:34:23.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.503+0000 7fb35801c740 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:59 5.7 5:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x5555b2a84000 con 0x5555b3442900 2024-09-17T04:34:23.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.504+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.5 v2:172.21.15.92:6826/248263832 11 ==== osd_op_reply(59 10000000001.00000000 [call] v41'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5555b273a000 con 0x5555b3442900 2024-09-17T04:34:23.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.504+0000 7fb35801c740 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:60 5.27 5:e4000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b2a84400 con 0x5555b3443680 2024-09-17T04:34:23.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.505+0000 7fb355d31640 1 -- 172.21.15.37:0/2608060258 <== osd.1 v2:172.21.15.37:6824/2657955935 8 ==== osd_op_reply(60 [pgnls start_epoch 41 out=49b] v33'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a780 con 0x5555b3443680 2024-09-17T04:34:23.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.505+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:61 5.17 5:e8000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3446c00 con 0x5555b27e4d00 2024-09-17T04:34:23.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.505+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.3 v2:172.21.15.37:6800/809975826 10 ==== osd_op_reply(61 [pgnls start_epoch 41 out=49b] v33'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a000 con 0x5555b27e4d00 2024-09-17T04:34:23.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.505+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:62 5.37 5:ec000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3598800 con 0x5555b3443200 2024-09-17T04:34:23.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.506+0000 7fb356532640 1 -- 172.21.15.37:0/2608060258 <== osd.4 v2:172.21.15.92:6818/2509785752 8 ==== osd_op_reply(62 [pgnls start_epoch 41 out=49b] v33'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a500 con 0x5555b3443200 2024-09-17T04:34:23.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.506+0000 7fb351528640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:63 5.f 5:f0000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3447000 con 0x5555b3442900 2024-09-17T04:34:23.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.506+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.5 v2:172.21.15.92:6826/248263832 12 ==== osd_op_reply(63 [pgnls start_epoch 41 out=49b] v33'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a000 con 0x5555b3442900 2024-09-17T04:34:23.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.506+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:64 5.2f 5:f4000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3599000 con 0x5555b3442000 2024-09-17T04:34:23.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.506+0000 7fb356532640 1 -- 172.21.15.37:0/2608060258 <== osd.7 v2:172.21.15.92:6802/3444787223 11 ==== osd_op_reply(64 [pgnls start_epoch 41 out=81b] v33'9 uv1 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5555b273a500 con 0x5555b3442000 2024-09-17T04:34:23.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.506+0000 7fb35801c740 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:65 5.2f 5:f526278d:::10000000001.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x5555b2a84800 con 0x5555b3442000 2024-09-17T04:34:23.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.507+0000 7fb356532640 1 -- 172.21.15.37:0/2608060258 <== osd.7 v2:172.21.15.92:6802/3444787223 12 ==== osd_op_reply(65 10000000001.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5555b273a500 con 0x5555b3442000 2024-09-17T04:34:23.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.507+0000 7fb35801c740 10 datascan.operator(): handling object 1099511627777.1 2024-09-17T04:34:23.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.507+0000 7fb35801c740 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:66 5.7 5:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x5555b3446400 con 0x5555b3442900 2024-09-17T04:34:23.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.508+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.5 v2:172.21.15.92:6826/248263832 13 ==== osd_op_reply(66 10000000001.00000000 [call] v41'10 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5555b273a000 con 0x5555b3442900 2024-09-17T04:34:23.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.508+0000 7fb35801c740 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] -- osd_op(unknown.0.0:67 5.1f 5:f8000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555b3446800 con 0x5555b3443b00 2024-09-17T04:34:23.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.508+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.6 v2:172.21.15.92:6810/4078365985 4 ==== osd_op_reply(67 [pgnls start_epoch 41 out=49b] v33'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a000 con 0x5555b3443b00 2024-09-17T04:34:23.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.508+0000 7fb350d27640 1 -- 172.21.15.37:0/2608060258 --> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] -- osd_op(unknown.0.0:68 5.3f 5:fc000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x5555ae60b800 con 0x5555b3443b00 2024-09-17T04:34:23.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 <== osd.6 v2:172.21.15.92:6810/4078365985 5 ==== osd_op_reply(68 [pgnls start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5555b273a000 con 0x5555b3443b00 2024-09-17T04:34:23.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb35801c740 1 -- 172.21.15.37:0/2608060258 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x5555b3443f80 msgr2=0x5555b2a5a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb35801c740 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x5555b3443f80 0x5555b2a5a680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb35801c740 1 -- 172.21.15.37:0/2608060258 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x5555b3443680 msgr2=0x5555b2a59b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb35801c740 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x5555b3443680 0x5555b2a59b80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb35801c740 1 -- 172.21.15.37:0/2608060258 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x5555b3442480 msgr2=0x5555b2a58580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb35801c740 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x5555b3442480 0x5555b2a58580 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb35801c740 1 -- 172.21.15.37:0/2608060258 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x5555b27e4d00 msgr2=0x5555b3412680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb35801c740 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x5555b27e4d00 0x5555b3412680 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 reap_dead start 2024-09-17T04:34:23.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb35801c740 1 -- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x5555b3443200 msgr2=0x5555b2a59600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb35801c740 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x5555b3443200 0x5555b2a59600 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 reap_dead start 2024-09-17T04:34:23.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb35801c740 1 -- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x5555b3442900 msgr2=0x5555b2a58b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb35801c740 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x5555b3442900 0x5555b2a58b00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb35801c740 1 -- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] conn(0x5555b3443b00 msgr2=0x5555b2a5a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb35801c740 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] conn(0x5555b3443b00 0x5555b2a5a100 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb35801c740 1 -- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x5555b3442000 msgr2=0x5555b2a58000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb35801c740 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x5555b3442000 0x5555b2a58000 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb35801c740 1 -- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x5555b27e5a80 msgr2=0x5555b3412100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb35801c740 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x5555b27e5a80 0x5555b3412100 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x5555b272f5c0 tx=0x5555b35430e0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb35801c740 1 -- 172.21.15.37:0/2608060258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555b27e2d80 msgr2=0x5555b3411080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb35801c740 1 --2- 172.21.15.37:0/2608060258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555b27e2d80 0x5555b3411080 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x5555b350f860 tx=0x5555b3542cc0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb355530640 1 -- 172.21.15.37:0/2608060258 reap_dead start 2024-09-17T04:34:23.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb35801c740 1 -- 172.21.15.37:0/2608060258 shutdown_connections 2024-09-17T04:34:23.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb35801c740 1 -- 172.21.15.37:0/2608060258 >> 172.21.15.37:0/2608060258 conn(0x5555b27e3200 msgr2=0x5555b275d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb35801c740 1 -- 172.21.15.37:0/2608060258 shutdown_connections 2024-09-17T04:34:23.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.509+0000 7fb35801c740 1 -- 172.21.15.37:0/2608060258 wait complete. 2024-09-17T04:34:23.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.510+0000 7fb35801c740 1 -- 172.21.15.37:0/2319491477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555b27e3680 msgr2=0x5555b3410580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.510+0000 7fb35801c740 1 --2- 172.21.15.37:0/2319491477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555b27e3680 0x5555b3410580 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x5555b350e0c0 tx=0x5555b273e960 comp rx=0 tx=0).stop 2024-09-17T04:34:23.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.510+0000 7fb35801c740 1 -- 172.21.15.37:0/2319491477 shutdown_connections 2024-09-17T04:34:23.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.510+0000 7fb35801c740 1 --2- 172.21.15.37:0/2319491477 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5555b27e3f80 0x5555b3410b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.510+0000 7fb35801c740 1 --2- 172.21.15.37:0/2319491477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555b27e3b00 0x5555b3410000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.510+0000 7fb35801c740 1 --2- 172.21.15.37:0/2319491477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555b27e3680 0x5555b3410580 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.510+0000 7fb35801c740 1 -- 172.21.15.37:0/2319491477 >> 172.21.15.37:0/2319491477 conn(0x5555b27e4400 msgr2=0x5555b275d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.510+0000 7fb35801c740 1 -- 172.21.15.37:0/2319491477 shutdown_connections 2024-09-17T04:34:23.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.510+0000 7fb35801c740 1 -- 172.21.15.37:0/2319491477 wait complete. 2024-09-17T04:34:23.519 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--alternate-pool', 'recovery_fs_metadata', '--filesystem', 'cephfs', 'cephfs_data'] in time 0:00:00.109112, result: 2024-09-17T04:34:23.519 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --alternate-pool recovery_fs_metadata --filesystem cephfs --force-corrupt --force-init cephfs_data 2024-09-17T04:34:23.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.580+0000 7f5ab4c0d740 1 -- start start 2024-09-17T04:34:23.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.580+0000 7f5ab4c0d740 1 Processor -- start 2024-09-17T04:34:23.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.581+0000 7f5ab4c0d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6289e1a80 0x55c629610580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.581+0000 7f5ab4c0d740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55c6289df200 0x55c629611080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.581+0000 7f5ab4c0d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6289ded80 0x55c629610b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.581+0000 7f5ab4c0d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c629622000 con 0x55c6289ded80 2024-09-17T04:34:23.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.581+0000 7f5ab4c0d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c629622180 con 0x55c6289e1a80 2024-09-17T04:34:23.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.581+0000 7f5ab4c0d740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x55c629622300 con 0x55c6289df200 2024-09-17T04:34:23.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.581+0000 7f5ab30b0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6289ded80 0x55c629610b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.581+0000 7f5ab30b0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6289ded80 0x55c629610b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38756/0 (socket says 172.21.15.37:38756) 2024-09-17T04:34:23.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.581+0000 7f5ab30b0640 1 -- 172.21.15.37:0/2961998030 learned_addr learned my addr 172.21.15.37:0/2961998030 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.581+0000 7f5ab20ae640 1 --2- 172.21.15.37:0/2961998030 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55c6289df200 0x55c629611080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.581+0000 7f5ab28af640 1 --2- 172.21.15.37:0/2961998030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6289e1a80 0x55c629610580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.581+0000 7f5ab30b0640 1 -- 172.21.15.37:0/2961998030 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55c6289df200 msgr2=0x55c629611080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.581+0000 7f5ab30b0640 1 --2- 172.21.15.37:0/2961998030 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55c6289df200 0x55c629611080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.581+0000 7f5ab30b0640 1 -- 172.21.15.37:0/2961998030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6289e1a80 msgr2=0x55c629610580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.581+0000 7f5ab30b0640 1 --2- 172.21.15.37:0/2961998030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6289e1a80 0x55c629610580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.581+0000 7f5ab30b0640 1 -- 172.21.15.37:0/2961998030 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c624833680 con 0x55c6289ded80 2024-09-17T04:34:23.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.581+0000 7f5ab30b0640 1 --2- 172.21.15.37:0/2961998030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6289ded80 0x55c629610b00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x55c62966fec0 tx=0x55c6288fef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f1d6eb78e4f783 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.582+0000 7f5ab00aa640 1 -- 172.21.15.37:0/2961998030 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c62482cb60 con 0x55c6289ded80 2024-09-17T04:34:23.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.582+0000 7f5ab00aa640 1 -- 172.21.15.37:0/2961998030 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55c62888cc40 con 0x55c6289ded80 2024-09-17T04:34:23.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.582+0000 7f5ab00aa640 1 -- 172.21.15.37:0/2961998030 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c62482cea0 con 0x55c6289ded80 2024-09-17T04:34:23.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.582+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/2961998030 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6248334a0 con 0x55c6289ded80 2024-09-17T04:34:23.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.582+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/2961998030 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55c624833a40 con 0x55c6289ded80 2024-09-17T04:34:23.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.582+0000 7f5ab4c0d740 4 waiting for MDS map... 2024-09-17T04:34:23.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.582+0000 7f5ab00aa640 1 -- 172.21.15.37:0/2961998030 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x55c62972a600 con 0x55c6289ded80 2024-09-17T04:34:23.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.582+0000 7f5ab00aa640 1 -- 172.21.15.37:0/2961998030 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 4143+0+0 (secure 0 0 0) 0x55c628891180 con 0x55c6289ded80 2024-09-17T04:34:23.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.582+0000 7f5ab4c0d740 4 Got MDS map 22 2024-09-17T04:34:23.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.582+0000 7f5ab4c0d740 4 datascan.main: Using metadata pool output 2024-09-17T04:34:23.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.582+0000 7f5ab4c0d740 4 datascan.main: connecting to RADOS... 2024-09-17T04:34:23.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.583+0000 7f5ab4c0d740 1 Processor -- start 2024-09-17T04:34:23.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.583+0000 7f5ab4c0d740 1 -- start start 2024-09-17T04:34:23.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.584+0000 7f5ab4c0d740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55c6289e0880 0x55c629610000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.584+0000 7f5ab4c0d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6289e0d00 0x55c629611600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.584+0000 7f5ab4c0d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6289e1180 0x55c629611b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.584+0000 7f5ab4c0d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c629622480 con 0x55c6289e1180 2024-09-17T04:34:23.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.584+0000 7f5ab4c0d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c629622600 con 0x55c6289e0d00 2024-09-17T04:34:23.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.584+0000 7f5ab4c0d740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x55c629622780 con 0x55c6289e0880 2024-09-17T04:34:23.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.584+0000 7f5ab20ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6289e1180 0x55c629611b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.584+0000 7f5ab20ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6289e1180 0x55c629611b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38764/0 (socket says 172.21.15.37:38764) 2024-09-17T04:34:23.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.584+0000 7f5ab20ae640 1 -- 172.21.15.37:0/2765944023 learned_addr learned my addr 172.21.15.37:0/2765944023 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.584+0000 7f5ab28af640 1 --2- 172.21.15.37:0/2765944023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6289e0d00 0x55c629611600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.584+0000 7f5ab20ae640 1 --2- 172.21.15.37:0/2765944023 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55c6289e0880 0x55c629610000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.585+0000 7f5ab20ae640 1 -- 172.21.15.37:0/2765944023 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55c6289e0880 msgr2=0x55c629610000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.585+0000 7f5ab20ae640 1 --2- 172.21.15.37:0/2765944023 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55c6289e0880 0x55c629610000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.585+0000 7f5ab20ae640 1 -- 172.21.15.37:0/2765944023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6289e0d00 msgr2=0x55c629611600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.585+0000 7f5ab20ae640 1 --2- 172.21.15.37:0/2765944023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6289e0d00 0x55c629611600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.585+0000 7f5ab20ae640 1 -- 172.21.15.37:0/2765944023 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6248323c0 con 0x55c6289e1180 2024-09-17T04:34:23.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.585+0000 7f5ab20ae640 1 --2- 172.21.15.37:0/2765944023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6289e1180 0x55c629611b80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55c629725410 tx=0x55c62971ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5956f1a55102925 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.585+0000 7f5aae0a6640 1 -- 172.21.15.37:0/2765944023 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c62482cb60 con 0x55c6289e1180 2024-09-17T04:34:23.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.585+0000 7f5aae0a6640 1 -- 172.21.15.37:0/2765944023 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55c62888c8c0 con 0x55c6289e1180 2024-09-17T04:34:23.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.585+0000 7f5aae0a6640 1 -- 172.21.15.37:0/2765944023 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c62482cea0 con 0x55c6289e1180 2024-09-17T04:34:23.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.585+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/2765944023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6289e1180 msgr2=0x55c629611b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.585+0000 7f5ab4c0d740 1 --2- 172.21.15.37:0/2765944023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6289e1180 0x55c629611b80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55c629725410 tx=0x55c62971ac60 comp rx=0 tx=0).stop 2024-09-17T04:34:23.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.585+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/2765944023 shutdown_connections 2024-09-17T04:34:23.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.585+0000 7f5ab4c0d740 1 --2- 172.21.15.37:0/2765944023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6289e1180 0x55c629611b80 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.585+0000 7f5ab4c0d740 1 --2- 172.21.15.37:0/2765944023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6289e0d00 0x55c629611600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.585+0000 7f5ab4c0d740 1 --2- 172.21.15.37:0/2765944023 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55c6289e0880 0x55c629610000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.585+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/2765944023 >> 172.21.15.37:0/2765944023 conn(0x55c6289de900 msgr2=0x55c628957000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.586+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/2765944023 shutdown_connections 2024-09-17T04:34:23.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.586+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/2765944023 wait complete. 2024-09-17T04:34:23.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.586+0000 7f5ab4c0d740 1 Processor -- start 2024-09-17T04:34:23.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.586+0000 7f5ab4c0d740 1 -- start start 2024-09-17T04:34:23.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.587+0000 7f5ab4c0d740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55c6289e0880 0x55c629610000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.587+0000 7f5ab4c0d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6289e0d00 0x55c629611600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.587+0000 7f5ab4c0d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6289e1180 0x55c629611b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.587+0000 7f5ab4c0d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c629622d80 con 0x55c6289e1180 2024-09-17T04:34:23.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.587+0000 7f5ab4c0d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c629622f00 con 0x55c6289e0d00 2024-09-17T04:34:23.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.587+0000 7f5ab4c0d740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x55c629623080 con 0x55c6289e0880 2024-09-17T04:34:23.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.588+0000 7f5ab20ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6289e1180 0x55c629611b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.588+0000 7f5ab20ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6289e1180 0x55c629611b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38780/0 (socket says 172.21.15.37:38780) 2024-09-17T04:34:23.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.588+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 learned_addr learned my addr 172.21.15.37:0/1380271642 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.588+0000 7f5ab28af640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6289e0d00 0x55c629611600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.588+0000 7f5ab20ae640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55c6289e0880 0x55c629610000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.588+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55c6289e0880 msgr2=0x55c629610000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.588+0000 7f5ab20ae640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55c6289e0880 0x55c629610000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.588+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6289e0d00 msgr2=0x55c629611600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.588+0000 7f5ab20ae640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6289e0d00 0x55c629611600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.588+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c624833680 con 0x55c6289e1180 2024-09-17T04:34:23.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.588+0000 7f5ab20ae640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6289e1180 0x55c629611b80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55c629796630 tx=0x55c62971aea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.588+0000 7f5aac8a3640 1 -- 172.21.15.37:0/1380271642 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c62482d1e0 con 0x55c6289e1180 2024-09-17T04:34:23.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.588+0000 7f5aac8a3640 1 -- 172.21.15.37:0/1380271642 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55c62888c000 con 0x55c6289e1180 2024-09-17T04:34:23.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.588+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c6248334a0 con 0x55c6289e1180 2024-09-17T04:34:23.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.588+0000 7f5aac8a3640 1 -- 172.21.15.37:0/1380271642 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c62482cb60 con 0x55c6289e1180 2024-09-17T04:34:23.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.588+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c624832b40 con 0x55c6289e1180 2024-09-17T04:34:23.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.588+0000 7f5ab4c0d740 4 datascan.init: forcing metadata pool 'recovery_fs_metadata' 2024-09-17T04:34:23.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.590+0000 7f5aac8a3640 1 -- 172.21.15.37:0/1380271642 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x55c62976a900 con 0x55c6289e1180 2024-09-17T04:34:23.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.590+0000 7f5aac8a3640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x55c6289dff80 0x55c629612100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.590+0000 7f5aac8a3640 1 -- 172.21.15.37:0/1380271642 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x55c629794a00 con 0x55c6289e1180 2024-09-17T04:34:23.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.590+0000 7f5ab4c0d740 4 datascan.main: data pool 'cephfs_data' has ID 5 2024-09-17T04:34:23.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.590+0000 7f5ab4c0d740 4 datascan.main: opening data pool 'cephfs_data' 2024-09-17T04:34:23.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.590+0000 7f5ab4c0d740 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x55c6289e1600 0x55c629612680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.590+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:1 6.14 6:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x55c62978ec00 con 0x55c6289e1600 2024-09-17T04:34:23.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.590+0000 7f5ab28af640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x55c6289dff80 0x55c629612100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.590+0000 7f5ab20ae640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x55c6289e1600 0x55c629612680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.591+0000 7f5ab28af640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x55c6289dff80 0x55c629612100 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55c62892b740 tx=0x55c62971b200 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.591+0000 7f5ab20ae640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x55c6289e1600 0x55c629612680 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.591+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.2 v2:172.21.15.37:6816/579720176 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv1 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x55c628938000 con 0x55c6289e1600 2024-09-17T04:34:23.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.591+0000 7f5ab4c0d740 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x55c628c52000 0x55c629612c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.591+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:2 6.1f 6:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x55c62978f000 con 0x55c628c52000 2024-09-17T04:34:23.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.591+0000 7f5ab30b0640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x55c628c52000 0x55c629612c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.592+0000 7f5ab30b0640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x55c628c52000 0x55c629612c00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.592+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.3 v2:172.21.15.37:6800/809975826 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv1 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x55c628938500 con 0x55c628c52000 2024-09-17T04:34:23.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.592+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:3 5.0 5:00000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c62978f400 con 0x55c628c52000 2024-09-17T04:34:23.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.592+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.3 v2:172.21.15.37:6800/809975826 2 ==== osd_op_reply(3 [pgnls-filter start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c52000 2024-09-17T04:34:23.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.592+0000 7f5aad8a5640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x55c628c55180 0x55c628c62000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.592+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:4 5.20 5:04000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c66000 con 0x55c628c55180 2024-09-17T04:34:23.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.593+0000 7f5ab28af640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x55c628c55180 0x55c628c62000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.593+0000 7f5ab28af640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x55c628c55180 0x55c628c62000 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.593+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.7 v2:172.21.15.92:6802/3444787223 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 41 out=49b] v33'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c55180 2024-09-17T04:34:23.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.593+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:5 5.10 5:08000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c66400 con 0x55c628c55180 2024-09-17T04:34:23.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.594+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.7 v2:172.21.15.92:6802/3444787223 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 41 out=49b] v33'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c55180 2024-09-17T04:34:23.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.594+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:6 5.30 5:0c000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c66800 con 0x55c628c55180 2024-09-17T04:34:23.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.594+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.7 v2:172.21.15.92:6802/3444787223 3 ==== osd_op_reply(6 [pgnls-filter start_epoch 41 out=49b] v33'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c55180 2024-09-17T04:34:23.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.594+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:7 5.8 5:10000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c67000 con 0x55c6289e1600 2024-09-17T04:34:23.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.594+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.2 v2:172.21.15.37:6816/579720176 2 ==== osd_op_reply(7 [pgnls-filter start_epoch 41 out=49b] v33'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c6289e1600 2024-09-17T04:34:23.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.594+0000 7f5aad8a5640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x55c628c55600 0x55c628c62580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.595+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:8 5.28 5:14000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c66c00 con 0x55c628c55600 2024-09-17T04:34:23.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.595+0000 7f5ab20ae640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x55c628c55600 0x55c628c62580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.595+0000 7f5ab20ae640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x55c628c55600 0x55c628c62580 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.595+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.5 v2:172.21.15.92:6826/248263832 1 ==== osd_op_reply(8 [pgnls-filter start_epoch 41 out=49b] v33'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c628c55600 2024-09-17T04:34:23.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.595+0000 7f5aae0a6640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x55c628c68000 0x55c628c63080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.595+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c67400 con 0x55c628c68000 2024-09-17T04:34:23.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.596+0000 7f5ab30b0640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x55c628c68000 0x55c628c63080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.596+0000 7f5ab30b0640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x55c628c68000 0x55c628c63080 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.596+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.4 v2:172.21.15.92:6818/2509785752 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c68000 2024-09-17T04:34:23.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.596+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:10 5.38 5:1c000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c67800 con 0x55c628c55180 2024-09-17T04:34:23.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.597+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.7 v2:172.21.15.92:6802/3444787223 4 ==== osd_op_reply(10 [pgnls-filter start_epoch 41 out=49b] v33'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c55180 2024-09-17T04:34:23.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.597+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:11 5.4 5:20000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c70000 con 0x55c628c55180 2024-09-17T04:34:23.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.597+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.7 v2:172.21.15.92:6802/3444787223 5 ==== osd_op_reply(11 [pgnls-filter start_epoch 41 out=49b] v33'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c55180 2024-09-17T04:34:23.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.597+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:12 5.24 5:24000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c67c00 con 0x55c628c55600 2024-09-17T04:34:23.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.598+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.5 v2:172.21.15.92:6826/248263832 2 ==== osd_op_reply(12 [pgnls-filter start_epoch 41 out=49b] v33'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c628c55600 2024-09-17T04:34:23.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.598+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:13 5.14 5:28000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c70400 con 0x55c628c52000 2024-09-17T04:34:23.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.598+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.3 v2:172.21.15.37:6800/809975826 3 ==== osd_op_reply(13 [pgnls-filter start_epoch 41 out=49b] v33'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c52000 2024-09-17T04:34:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.598+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:14 5.34 5:2c000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c62978f000 con 0x55c628c55600 2024-09-17T04:34:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.599+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.5 v2:172.21.15.92:6826/248263832 3 ==== osd_op_reply(14 [pgnls-filter start_epoch 41 out=49b] v33'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c628c55600 2024-09-17T04:34:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.599+0000 7f5aae0a6640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x55c628c68480 0x55c628c63600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.599+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:15 5.c 5:30000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c70800 con 0x55c628c68480 2024-09-17T04:34:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.599+0000 7f5ab28af640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x55c628c68480 0x55c628c63600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.599+0000 7f5ab28af640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x55c628c68480 0x55c628c63600 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.599+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.1 v2:172.21.15.37:6824/2657955935 1 ==== osd_op_reply(15 [pgnls-filter start_epoch 41 out=49b] v33'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c68480 2024-09-17T04:34:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.599+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:16 5.2c 5:34000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c62978f400 con 0x55c6289e1600 2024-09-17T04:34:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.600+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.2 v2:172.21.15.37:6816/579720176 3 ==== osd_op_reply(16 [pgnls-filter start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c6289e1600 2024-09-17T04:34:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.600+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:17 5.1c 5:38000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c70400 con 0x55c628c68000 2024-09-17T04:34:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.600+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.4 v2:172.21.15.92:6818/2509785752 2 ==== osd_op_reply(17 [pgnls-filter start_epoch 41 out=49b] v33'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c68000 2024-09-17T04:34:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.600+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:18 5.3c 5:3c000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c62978ec00 con 0x55c628c68480 2024-09-17T04:34:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.600+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.1 v2:172.21.15.37:6824/2657955935 2 ==== osd_op_reply(18 [pgnls-filter start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c68480 2024-09-17T04:34:23.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.600+0000 7f5aae0a6640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] conn(0x55c628c68900 0x55c628c63b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.600+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] -- osd_op(unknown.0.0:19 5.2 5:40000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c70000 con 0x55c628c68900 2024-09-17T04:34:23.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.601+0000 7f5ab20ae640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] conn(0x55c628c68900 0x55c628c63b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.601+0000 7f5ab20ae640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] conn(0x55c628c68900 0x55c628c63b80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.601+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.6 v2:172.21.15.92:6810/4078365985 1 ==== osd_op_reply(19 [pgnls-filter start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c628c68900 2024-09-17T04:34:23.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.601+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:20 5.22 5:44000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c629729400 con 0x55c628c55180 2024-09-17T04:34:23.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.602+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.7 v2:172.21.15.92:6802/3444787223 6 ==== osd_op_reply(20 [pgnls-filter start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c55180 2024-09-17T04:34:23.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.602+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:21 5.12 5:48000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c70c00 con 0x55c628c68480 2024-09-17T04:34:23.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.602+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.1 v2:172.21.15.37:6824/2657955935 3 ==== osd_op_reply(21 [pgnls-filter start_epoch 41 out=49b] v33'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c68480 2024-09-17T04:34:23.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.602+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:22 5.32 5:4c000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c71400 con 0x55c628c52000 2024-09-17T04:34:23.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.602+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.3 v2:172.21.15.37:6800/809975826 4 ==== osd_op_reply(22 [pgnls-filter start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c52000 2024-09-17T04:34:23.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.602+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:23 5.a 5:50000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c71000 con 0x55c6289e1600 2024-09-17T04:34:23.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.602+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.2 v2:172.21.15.37:6816/579720176 4 ==== osd_op_reply(23 [pgnls-filter start_epoch 41 out=49b] v33'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c6289e1600 2024-09-17T04:34:23.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.602+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:24 5.2a 5:54000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c71800 con 0x55c628c68000 2024-09-17T04:34:23.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.603+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.4 v2:172.21.15.92:6818/2509785752 3 ==== osd_op_reply(24 [pgnls-filter start_epoch 41 out=49b] v33'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c68000 2024-09-17T04:34:23.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.603+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:25 5.1a 5:58000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c66000 con 0x55c628c55180 2024-09-17T04:34:23.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.603+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.7 v2:172.21.15.92:6802/3444787223 7 ==== osd_op_reply(25 [pgnls-filter start_epoch 41 out=49b] v33'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c55180 2024-09-17T04:34:23.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.603+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:26 5.3a 5:5c000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c71c00 con 0x55c628c52000 2024-09-17T04:34:23.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.603+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.3 v2:172.21.15.37:6800/809975826 5 ==== osd_op_reply(26 [pgnls-filter start_epoch 41 out=49b] v33'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c52000 2024-09-17T04:34:23.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.604+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:27 5.6 5:60000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c66400 con 0x55c6289e1600 2024-09-17T04:34:23.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.604+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.2 v2:172.21.15.37:6816/579720176 5 ==== osd_op_reply(27 [pgnls-filter start_epoch 41 out=49b] v33'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c6289e1600 2024-09-17T04:34:23.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.604+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:28 5.26 5:64000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c66400 con 0x55c628c52000 2024-09-17T04:34:23.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.604+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.3 v2:172.21.15.37:6800/809975826 6 ==== osd_op_reply(28 [pgnls-filter start_epoch 41 out=49b] v33'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c52000 2024-09-17T04:34:23.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.604+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:29 5.16 5:68000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c66800 con 0x55c628c55600 2024-09-17T04:34:23.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.604+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.5 v2:172.21.15.92:6826/248263832 4 ==== osd_op_reply(29 [pgnls-filter start_epoch 41 out=49b] v33'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c628c55600 2024-09-17T04:34:23.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.604+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:30 5.36 5:6c000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c67400 con 0x55c628c68480 2024-09-17T04:34:23.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.605+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.1 v2:172.21.15.37:6824/2657955935 4 ==== osd_op_reply(30 [pgnls-filter start_epoch 41 out=49b] v33'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c68480 2024-09-17T04:34:23.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.605+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:31 5.e 5:70000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c66c00 con 0x55c628c68000 2024-09-17T04:34:23.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.605+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.4 v2:172.21.15.92:6818/2509785752 4 ==== osd_op_reply(31 [pgnls-filter start_epoch 41 out=49b] v33'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c68000 2024-09-17T04:34:23.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.605+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:32 5.2e 5:74000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c70000 con 0x55c628c52000 2024-09-17T04:34:23.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.605+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.3 v2:172.21.15.37:6800/809975826 7 ==== osd_op_reply(32 [pgnls-filter start_epoch 41 out=49b] v33'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c52000 2024-09-17T04:34:23.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.605+0000 7f5aae0a6640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x55c628c68d80 0x55c628c64100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.605+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:33 5.1e 5:78000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c67000 con 0x55c628c68d80 2024-09-17T04:34:23.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.606+0000 7f5ab30b0640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x55c628c68d80 0x55c628c64100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.606+0000 7f5ab30b0640 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x55c628c68d80 0x55c628c64100 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.606+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.0 v2:172.21.15.37:6808/2283587593 1 ==== osd_op_reply(33 [pgnls-filter start_epoch 41 out=49b] v33'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c68d80 2024-09-17T04:34:23.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.606+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:34 5.3e 5:7c000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c71000 con 0x55c628c52000 2024-09-17T04:34:23.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.606+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.3 v2:172.21.15.37:6800/809975826 8 ==== osd_op_reply(34 [pgnls-filter start_epoch 41 out=49b] v33'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c52000 2024-09-17T04:34:23.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.606+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:35 5.1 5:80000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c70400 con 0x55c628c68000 2024-09-17T04:34:23.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.607+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.4 v2:172.21.15.92:6818/2509785752 5 ==== osd_op_reply(35 [pgnls-filter start_epoch 41 out=49b] v33'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c68000 2024-09-17T04:34:23.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.607+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] -- osd_op(unknown.0.0:36 5.21 5:84000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c62978f000 con 0x55c628c68900 2024-09-17T04:34:23.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.607+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.6 v2:172.21.15.92:6810/4078365985 2 ==== osd_op_reply(36 [pgnls-filter start_epoch 41 out=49b] v33'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c628c68900 2024-09-17T04:34:23.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.607+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] -- osd_op(unknown.0.0:37 5.11 5:88000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c71400 con 0x55c628c68900 2024-09-17T04:34:23.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.607+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.6 v2:172.21.15.92:6810/4078365985 3 ==== osd_op_reply(37 [pgnls-filter start_epoch 41 out=49b] v33'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c628c68900 2024-09-17T04:34:23.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.607+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:38 5.31 5:8c000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c62978f400 con 0x55c628c55600 2024-09-17T04:34:23.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.608+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.5 v2:172.21.15.92:6826/248263832 5 ==== osd_op_reply(38 [pgnls-filter start_epoch 41 out=49b] v33'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c628c55600 2024-09-17T04:34:23.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.608+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:39 5.9 5:90000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c71800 con 0x55c628c55180 2024-09-17T04:34:23.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.608+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.7 v2:172.21.15.92:6802/3444787223 8 ==== osd_op_reply(39 [pgnls-filter start_epoch 41 out=49b] v33'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c55180 2024-09-17T04:34:23.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.608+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:40 5.29 5:94000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c66400 con 0x55c628c68d80 2024-09-17T04:34:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.609+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.0 v2:172.21.15.37:6808/2283587593 2 ==== osd_op_reply(40 [pgnls-filter start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c68d80 2024-09-17T04:34:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.609+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:41 5.19 5:98000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c71c00 con 0x55c628c68480 2024-09-17T04:34:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.609+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.1 v2:172.21.15.37:6824/2657955935 5 ==== osd_op_reply(41 [pgnls-filter start_epoch 41 out=49b] v33'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c68480 2024-09-17T04:34:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.609+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:42 5.39 5:9c000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c66c00 con 0x55c628c55180 2024-09-17T04:34:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.609+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.7 v2:172.21.15.92:6802/3444787223 9 ==== osd_op_reply(42 [pgnls-filter start_epoch 41 out=49b] v33'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c55180 2024-09-17T04:34:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.609+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:43 5.5 5:a0000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c629729400 con 0x55c628c68d80 2024-09-17T04:34:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.610+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.0 v2:172.21.15.37:6808/2283587593 3 ==== osd_op_reply(43 [pgnls-filter start_epoch 41 out=49b] v33'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c68d80 2024-09-17T04:34:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.610+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:44 5.25 5:a4000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c66000 con 0x55c628c68480 2024-09-17T04:34:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.610+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.1 v2:172.21.15.37:6824/2657955935 6 ==== osd_op_reply(44 [pgnls-filter start_epoch 41 out=49b] v33'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c68480 2024-09-17T04:34:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.610+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:45 5.15 5:a8000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c62978ec00 con 0x55c628c55600 2024-09-17T04:34:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.610+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.5 v2:172.21.15.92:6826/248263832 6 ==== osd_op_reply(45 [pgnls-filter start_epoch 41 out=49b] v33'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c628c55600 2024-09-17T04:34:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.610+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:46 5.35 5:ac000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c67400 con 0x55c628c55180 2024-09-17T04:34:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.611+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.7 v2:172.21.15.92:6802/3444787223 10 ==== osd_op_reply(46 [pgnls-filter start_epoch 41 out=49b] v33'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c55180 2024-09-17T04:34:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.611+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:47 5.d 5:b0000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c67c00 con 0x55c6289e1600 2024-09-17T04:34:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.611+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.2 v2:172.21.15.37:6816/579720176 6 ==== osd_op_reply(47 [pgnls-filter start_epoch 41 out=49b] v33'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c6289e1600 2024-09-17T04:34:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.611+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:48 5.2d 5:b4000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c67800 con 0x55c628c52000 2024-09-17T04:34:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.611+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.3 v2:172.21.15.37:6800/809975826 9 ==== osd_op_reply(48 [pgnls-filter start_epoch 41 out=49b] v33'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c52000 2024-09-17T04:34:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.611+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:49 5.1d 5:b8000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c62978f000 con 0x55c628c68480 2024-09-17T04:34:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.612+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.1 v2:172.21.15.37:6824/2657955935 7 ==== osd_op_reply(49 [pgnls-filter start_epoch 41 out=49b] v33'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c68480 2024-09-17T04:34:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.612+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:50 5.3d 5:bc000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c70800 con 0x55c628c55600 2024-09-17T04:34:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.612+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.5 v2:172.21.15.92:6826/248263832 7 ==== osd_op_reply(50 [pgnls-filter start_epoch 41 out=49b] v33'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c628c55600 2024-09-17T04:34:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.612+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:51 5.3 5:c0000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c62978f400 con 0x55c628c68d80 2024-09-17T04:34:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.612+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.0 v2:172.21.15.37:6808/2283587593 4 ==== osd_op_reply(51 [pgnls-filter start_epoch 41 out=49b] v33'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c68d80 2024-09-17T04:34:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.612+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:52 5.23 5:c4000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c70c00 con 0x55c628c68000 2024-09-17T04:34:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.613+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.4 v2:172.21.15.92:6818/2509785752 6 ==== osd_op_reply(52 [pgnls-filter start_epoch 41 out=49b] v33'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c68000 2024-09-17T04:34:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.613+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:53 5.13 5:c8000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c62978ec00 con 0x55c628c52000 2024-09-17T04:34:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.613+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.3 v2:172.21.15.37:6800/809975826 10 ==== osd_op_reply(53 [pgnls-filter start_epoch 41 out=49b] v33'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c52000 2024-09-17T04:34:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.613+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:54 5.33 5:cc000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c67000 con 0x55c6289e1600 2024-09-17T04:34:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.613+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.2 v2:172.21.15.37:6816/579720176 7 ==== osd_op_reply(54 [pgnls-filter start_epoch 41 out=49b] v33'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c6289e1600 2024-09-17T04:34:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.613+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:55 5.b 5:d0000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c70000 con 0x55c6289e1600 2024-09-17T04:34:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.613+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.2 v2:172.21.15.37:6816/579720176 8 ==== osd_op_reply(55 [pgnls-filter start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c6289e1600 2024-09-17T04:34:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.613+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:56 5.2b 5:d4000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c66400 con 0x55c628c68000 2024-09-17T04:34:23.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.614+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.4 v2:172.21.15.92:6818/2509785752 7 ==== osd_op_reply(56 [pgnls-filter start_epoch 41 out=49b] v33'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c68000 2024-09-17T04:34:23.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.614+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:57 5.1b 5:d8000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c71000 con 0x55c628c55600 2024-09-17T04:34:23.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.614+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.5 v2:172.21.15.92:6826/248263832 8 ==== osd_op_reply(57 [pgnls-filter start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c628c55600 2024-09-17T04:34:23.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.614+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:58 5.3b 5:dc000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c67800 con 0x55c6289e1600 2024-09-17T04:34:23.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.614+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.2 v2:172.21.15.37:6816/579720176 9 ==== osd_op_reply(58 [pgnls-filter start_epoch 41 out=49b] v33'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c6289e1600 2024-09-17T04:34:23.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.614+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:59 5.7 5:e0000000::::head [pgnls-filter start_epoch 41 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c70400 con 0x55c628c55600 2024-09-17T04:34:23.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.615+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.5 v2:172.21.15.92:6826/248263832 9 ==== osd_op_reply(59 [pgnls-filter start_epoch 41 out=81b] v41'10 uv10 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55c628938000 con 0x55c628c55600 2024-09-17T04:34:23.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.615+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:60 5.0 5:00000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c62978f800 con 0x55c628c52000 2024-09-17T04:34:23.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.615+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.3 v2:172.21.15.37:6800/809975826 11 ==== osd_op_reply(60 [pgnls-filter start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c52000 2024-09-17T04:34:23.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.615+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:61 5.20 5:04000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c71400 con 0x55c628c55180 2024-09-17T04:34:23.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.616+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.7 v2:172.21.15.92:6802/3444787223 11 ==== osd_op_reply(61 [pgnls-filter start_epoch 41 out=49b] v33'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c55180 2024-09-17T04:34:23.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.616+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:62 5.10 5:08000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c66800 con 0x55c628c55180 2024-09-17T04:34:23.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.616+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.7 v2:172.21.15.92:6802/3444787223 12 ==== osd_op_reply(62 [pgnls-filter start_epoch 41 out=49b] v33'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c55180 2024-09-17T04:34:23.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.616+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:63 5.30 5:0c000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c629729400 con 0x55c628c55180 2024-09-17T04:34:23.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.616+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.7 v2:172.21.15.92:6802/3444787223 13 ==== osd_op_reply(63 [pgnls-filter start_epoch 41 out=49b] v33'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c55180 2024-09-17T04:34:23.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.616+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:64 5.8 5:10000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c67c00 con 0x55c6289e1600 2024-09-17T04:34:23.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.617+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.2 v2:172.21.15.37:6816/579720176 10 ==== osd_op_reply(64 [pgnls-filter start_epoch 41 out=49b] v33'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c6289e1600 2024-09-17T04:34:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.617+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:65 5.28 5:14000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c70800 con 0x55c628c55600 2024-09-17T04:34:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.617+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.5 v2:172.21.15.92:6826/248263832 10 ==== osd_op_reply(65 [pgnls-filter start_epoch 41 out=49b] v33'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c628c55600 2024-09-17T04:34:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.617+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:66 5.18 5:18000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c71000 con 0x55c628c68000 2024-09-17T04:34:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.618+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.4 v2:172.21.15.92:6818/2509785752 8 ==== osd_op_reply(66 [pgnls-filter start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c68000 2024-09-17T04:34:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.618+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:67 5.38 5:1c000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c70000 con 0x55c628c55180 2024-09-17T04:34:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.618+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.7 v2:172.21.15.92:6802/3444787223 14 ==== osd_op_reply(67 [pgnls-filter start_epoch 41 out=49b] v33'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c55180 2024-09-17T04:34:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.618+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:68 5.4 5:20000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c70400 con 0x55c628c55180 2024-09-17T04:34:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.619+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.7 v2:172.21.15.92:6802/3444787223 15 ==== osd_op_reply(68 [pgnls-filter start_epoch 41 out=49b] v33'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c55180 2024-09-17T04:34:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.619+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:69 5.24 5:24000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c62978f000 con 0x55c628c55600 2024-09-17T04:34:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.619+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.5 v2:172.21.15.92:6826/248263832 11 ==== osd_op_reply(69 [pgnls-filter start_epoch 41 out=49b] v33'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c628c55600 2024-09-17T04:34:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.619+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:70 5.14 5:28000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c71800 con 0x55c628c52000 2024-09-17T04:34:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.620+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.3 v2:172.21.15.37:6800/809975826 12 ==== osd_op_reply(70 [pgnls-filter start_epoch 41 out=49b] v33'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c52000 2024-09-17T04:34:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.620+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:71 5.34 5:2c000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c67000 con 0x55c628c55600 2024-09-17T04:34:23.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.620+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.5 v2:172.21.15.92:6826/248263832 12 ==== osd_op_reply(71 [pgnls-filter start_epoch 41 out=49b] v33'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c628c55600 2024-09-17T04:34:23.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.620+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:72 5.c 5:30000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c71c00 con 0x55c628c68480 2024-09-17T04:34:23.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.620+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.1 v2:172.21.15.37:6824/2657955935 8 ==== osd_op_reply(72 [pgnls-filter start_epoch 41 out=49b] v33'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c68480 2024-09-17T04:34:23.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.620+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:73 5.2c 5:34000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c67800 con 0x55c6289e1600 2024-09-17T04:34:23.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.621+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.2 v2:172.21.15.37:6816/579720176 11 ==== osd_op_reply(73 [pgnls-filter start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c6289e1600 2024-09-17T04:34:23.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.621+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:74 5.1c 5:38000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c71400 con 0x55c628c68000 2024-09-17T04:34:23.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.621+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.4 v2:172.21.15.92:6818/2509785752 9 ==== osd_op_reply(74 [pgnls-filter start_epoch 41 out=49b] v33'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c68000 2024-09-17T04:34:23.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.621+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:75 5.3c 5:3c000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c67c00 con 0x55c628c68480 2024-09-17T04:34:23.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.621+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.1 v2:172.21.15.37:6824/2657955935 9 ==== osd_op_reply(75 [pgnls-filter start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c68480 2024-09-17T04:34:23.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.621+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] -- osd_op(unknown.0.0:76 5.2 5:40000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c70c00 con 0x55c628c68900 2024-09-17T04:34:23.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.622+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.6 v2:172.21.15.92:6810/4078365985 4 ==== osd_op_reply(76 [pgnls-filter start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c628c68900 2024-09-17T04:34:23.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.622+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:77 5.22 5:44000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c66c00 con 0x55c628c55180 2024-09-17T04:34:23.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.622+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.7 v2:172.21.15.92:6802/3444787223 16 ==== osd_op_reply(77 [pgnls-filter start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c55180 2024-09-17T04:34:23.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.622+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:78 5.12 5:48000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c71000 con 0x55c628c68480 2024-09-17T04:34:23.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.622+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.1 v2:172.21.15.37:6824/2657955935 10 ==== osd_op_reply(78 [pgnls-filter start_epoch 41 out=49b] v33'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c68480 2024-09-17T04:34:23.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.622+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:79 5.32 5:4c000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c66000 con 0x55c628c52000 2024-09-17T04:34:23.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.622+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.3 v2:172.21.15.37:6800/809975826 13 ==== osd_op_reply(79 [pgnls-filter start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c52000 2024-09-17T04:34:23.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.623+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:80 5.a 5:50000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c71800 con 0x55c6289e1600 2024-09-17T04:34:23.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.623+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.2 v2:172.21.15.37:6816/579720176 12 ==== osd_op_reply(80 [pgnls-filter start_epoch 41 out=49b] v33'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c6289e1600 2024-09-17T04:34:23.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.623+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:81 5.2a 5:54000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c67400 con 0x55c628c68000 2024-09-17T04:34:23.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.623+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.4 v2:172.21.15.92:6818/2509785752 10 ==== osd_op_reply(81 [pgnls-filter start_epoch 41 out=49b] v33'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c68000 2024-09-17T04:34:23.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.623+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:82 5.1a 5:58000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c71400 con 0x55c628c55180 2024-09-17T04:34:23.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.623+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.7 v2:172.21.15.92:6802/3444787223 17 ==== osd_op_reply(82 [pgnls-filter start_epoch 41 out=49b] v33'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c55180 2024-09-17T04:34:23.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.623+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:83 5.3a 5:5c000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c70000 con 0x55c628c52000 2024-09-17T04:34:23.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.624+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.3 v2:172.21.15.37:6800/809975826 14 ==== osd_op_reply(83 [pgnls-filter start_epoch 41 out=49b] v33'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c52000 2024-09-17T04:34:23.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.624+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:84 5.6 5:60000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c62978f400 con 0x55c6289e1600 2024-09-17T04:34:23.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.624+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.2 v2:172.21.15.37:6816/579720176 13 ==== osd_op_reply(84 [pgnls-filter start_epoch 41 out=49b] v33'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c6289e1600 2024-09-17T04:34:23.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.624+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:85 5.26 5:64000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c70400 con 0x55c628c52000 2024-09-17T04:34:23.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.624+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.3 v2:172.21.15.37:6800/809975826 15 ==== osd_op_reply(85 [pgnls-filter start_epoch 41 out=49b] v33'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c52000 2024-09-17T04:34:23.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.624+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:86 5.16 5:68000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c62978ec00 con 0x55c628c55600 2024-09-17T04:34:23.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.625+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.5 v2:172.21.15.92:6826/248263832 13 ==== osd_op_reply(86 [pgnls-filter start_epoch 41 out=49b] v33'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c628c55600 2024-09-17T04:34:23.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.625+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:87 5.36 5:6c000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c71c00 con 0x55c628c68480 2024-09-17T04:34:23.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.625+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.1 v2:172.21.15.37:6824/2657955935 11 ==== osd_op_reply(87 [pgnls-filter start_epoch 41 out=49b] v33'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c68480 2024-09-17T04:34:23.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.625+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:88 5.e 5:70000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c62978f800 con 0x55c628c68000 2024-09-17T04:34:23.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.625+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.4 v2:172.21.15.92:6818/2509785752 11 ==== osd_op_reply(88 [pgnls-filter start_epoch 41 out=49b] v33'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c68000 2024-09-17T04:34:23.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.625+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:89 5.2e 5:74000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c71000 con 0x55c628c52000 2024-09-17T04:34:23.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.626+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.3 v2:172.21.15.37:6800/809975826 16 ==== osd_op_reply(89 [pgnls-filter start_epoch 41 out=49b] v33'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c52000 2024-09-17T04:34:23.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.626+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:90 5.1e 5:78000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c66400 con 0x55c628c68d80 2024-09-17T04:34:23.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.626+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.0 v2:172.21.15.37:6808/2283587593 5 ==== osd_op_reply(90 [pgnls-filter start_epoch 41 out=49b] v33'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c68d80 2024-09-17T04:34:23.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.626+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:91 5.3e 5:7c000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c66800 con 0x55c628c52000 2024-09-17T04:34:23.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.626+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.3 v2:172.21.15.37:6800/809975826 17 ==== osd_op_reply(91 [pgnls-filter start_epoch 41 out=49b] v33'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c52000 2024-09-17T04:34:23.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.626+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:92 5.1 5:80000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c62978f000 con 0x55c628c68000 2024-09-17T04:34:23.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.626+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.4 v2:172.21.15.92:6818/2509785752 12 ==== osd_op_reply(92 [pgnls-filter start_epoch 41 out=49b] v33'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c68000 2024-09-17T04:34:23.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.626+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] -- osd_op(unknown.0.0:93 5.21 5:84000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c67c00 con 0x55c628c68900 2024-09-17T04:34:23.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.627+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.6 v2:172.21.15.92:6810/4078365985 5 ==== osd_op_reply(93 [pgnls-filter start_epoch 41 out=49b] v33'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c628c68900 2024-09-17T04:34:23.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.627+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] -- osd_op(unknown.0.0:94 5.11 5:88000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c62978f400 con 0x55c628c68900 2024-09-17T04:34:23.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.627+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.6 v2:172.21.15.92:6810/4078365985 6 ==== osd_op_reply(94 [pgnls-filter start_epoch 41 out=49b] v33'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c628c68900 2024-09-17T04:34:23.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.627+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:95 5.31 5:8c000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c66c00 con 0x55c628c55600 2024-09-17T04:34:23.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.627+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.5 v2:172.21.15.92:6826/248263832 14 ==== osd_op_reply(95 [pgnls-filter start_epoch 41 out=49b] v33'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c628c55600 2024-09-17T04:34:23.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.627+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:96 5.9 5:90000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c62978ec00 con 0x55c628c55180 2024-09-17T04:34:23.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.628+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.7 v2:172.21.15.92:6802/3444787223 18 ==== osd_op_reply(96 [pgnls-filter start_epoch 41 out=49b] v33'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c55180 2024-09-17T04:34:23.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.628+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:97 5.29 5:94000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c629729400 con 0x55c628c68d80 2024-09-17T04:34:23.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.628+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.0 v2:172.21.15.37:6808/2283587593 6 ==== osd_op_reply(97 [pgnls-filter start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c68d80 2024-09-17T04:34:23.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.628+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:98 5.19 5:98000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c70800 con 0x55c628c68480 2024-09-17T04:34:23.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.629+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.1 v2:172.21.15.37:6824/2657955935 12 ==== osd_op_reply(98 [pgnls-filter start_epoch 41 out=49b] v33'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c68480 2024-09-17T04:34:23.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.629+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:99 5.39 5:9c000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c62978f800 con 0x55c628c55180 2024-09-17T04:34:23.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.629+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.7 v2:172.21.15.92:6802/3444787223 19 ==== osd_op_reply(99 [pgnls-filter start_epoch 41 out=49b] v33'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c55180 2024-09-17T04:34:23.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.629+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:100 5.5 5:a0000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c70c00 con 0x55c628c68d80 2024-09-17T04:34:23.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.629+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.0 v2:172.21.15.37:6808/2283587593 7 ==== osd_op_reply(100 [pgnls-filter start_epoch 41 out=49b] v33'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c68d80 2024-09-17T04:34:23.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.629+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:101 5.25 5:a4000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c62978f000 con 0x55c628c68480 2024-09-17T04:34:23.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.630+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.1 v2:172.21.15.37:6824/2657955935 13 ==== osd_op_reply(101 [pgnls-filter start_epoch 41 out=49b] v33'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c68480 2024-09-17T04:34:23.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.630+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:102 5.15 5:a8000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c71800 con 0x55c628c55600 2024-09-17T04:34:23.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.630+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.5 v2:172.21.15.92:6826/248263832 15 ==== osd_op_reply(102 [pgnls-filter start_epoch 41 out=49b] v33'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c628c55600 2024-09-17T04:34:23.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.630+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:103 5.35 5:ac000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c66000 con 0x55c628c55180 2024-09-17T04:34:23.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.630+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.7 v2:172.21.15.92:6802/3444787223 20 ==== osd_op_reply(103 [pgnls-filter start_epoch 41 out=49b] v33'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c55180 2024-09-17T04:34:23.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.630+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:104 5.d 5:b0000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c67000 con 0x55c6289e1600 2024-09-17T04:34:23.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.631+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.2 v2:172.21.15.37:6816/579720176 14 ==== osd_op_reply(104 [pgnls-filter start_epoch 41 out=49b] v33'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c6289e1600 2024-09-17T04:34:23.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.631+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:105 5.2d 5:b4000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c67400 con 0x55c628c52000 2024-09-17T04:34:23.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.631+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.3 v2:172.21.15.37:6800/809975826 18 ==== osd_op_reply(105 [pgnls-filter start_epoch 41 out=49b] v33'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c52000 2024-09-17T04:34:23.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.631+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:106 5.1d 5:b8000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c67800 con 0x55c628c68480 2024-09-17T04:34:23.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.631+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.1 v2:172.21.15.37:6824/2657955935 14 ==== osd_op_reply(106 [pgnls-filter start_epoch 41 out=49b] v33'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c68480 2024-09-17T04:34:23.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.631+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:107 5.3d 5:bc000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c66400 con 0x55c628c55600 2024-09-17T04:34:23.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.632+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.5 v2:172.21.15.92:6826/248263832 16 ==== osd_op_reply(107 [pgnls-filter start_epoch 41 out=49b] v33'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c628c55600 2024-09-17T04:34:23.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.632+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:108 5.3 5:c0000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c92000 con 0x55c628c68d80 2024-09-17T04:34:23.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.632+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.0 v2:172.21.15.37:6808/2283587593 8 ==== osd_op_reply(108 [pgnls-filter start_epoch 41 out=49b] v33'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c68d80 2024-09-17T04:34:23.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.632+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:109 5.23 5:c4000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c66800 con 0x55c628c68000 2024-09-17T04:34:23.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.632+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.4 v2:172.21.15.92:6818/2509785752 13 ==== osd_op_reply(109 [pgnls-filter start_epoch 41 out=49b] v33'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c68000 2024-09-17T04:34:23.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.632+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:110 5.13 5:c8000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c92400 con 0x55c628c52000 2024-09-17T04:34:23.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.633+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.3 v2:172.21.15.37:6800/809975826 19 ==== osd_op_reply(110 [pgnls-filter start_epoch 41 out=49b] v33'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c52000 2024-09-17T04:34:23.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.633+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:111 5.33 5:cc000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c70000 con 0x55c6289e1600 2024-09-17T04:34:23.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.633+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.2 v2:172.21.15.37:6816/579720176 15 ==== osd_op_reply(111 [pgnls-filter start_epoch 41 out=49b] v33'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c6289e1600 2024-09-17T04:34:23.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.633+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:112 5.b 5:d0000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c92800 con 0x55c6289e1600 2024-09-17T04:34:23.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.633+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.2 v2:172.21.15.37:6816/579720176 16 ==== osd_op_reply(112 [pgnls-filter start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c6289e1600 2024-09-17T04:34:23.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.633+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:113 5.2b 5:d4000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c70400 con 0x55c628c68000 2024-09-17T04:34:23.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.633+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.4 v2:172.21.15.92:6818/2509785752 14 ==== osd_op_reply(113 [pgnls-filter start_epoch 41 out=49b] v33'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938500 con 0x55c628c68000 2024-09-17T04:34:23.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.633+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:114 5.1b 5:d8000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c92c00 con 0x55c628c55600 2024-09-17T04:34:23.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.634+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.5 v2:172.21.15.92:6826/248263832 17 ==== osd_op_reply(114 [pgnls-filter start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c628c55600 2024-09-17T04:34:23.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.634+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:115 5.3b 5:dc000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c71000 con 0x55c6289e1600 2024-09-17T04:34:23.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.634+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.2 v2:172.21.15.37:6816/579720176 17 ==== osd_op_reply(115 [pgnls-filter start_epoch 41 out=49b] v33'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c6289e1600 2024-09-17T04:34:23.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.634+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:116 5.7 5:e0000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c93000 con 0x55c628c55600 2024-09-17T04:34:23.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.634+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.5 v2:172.21.15.92:6826/248263832 18 ==== osd_op_reply(116 [pgnls-filter start_epoch 41 out=81b] v41'10 uv10 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55c628938000 con 0x55c628c55600 2024-09-17T04:34:23.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.634+0000 7f5ab4c0d740 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-09-17T04:34:23.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.634+0000 7f5ab4c0d740 10 datascan.operator(): handling object 10000000001.0 2024-09-17T04:34:23.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.634+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:117 5.7 5:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x55c628c67c00 con 0x55c628c55600 2024-09-17T04:34:23.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.635+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.5 v2:172.21.15.92:6826/248263832 19 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (95) out=95b,getxattr (30) out=30b,getxattr] v0'0 uv10 ondisk = 0) ==== 416+0+163 (crc 0 0 0) 0x55c628938000 con 0x55c628c55600 2024-09-17T04:34:23.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.635+0000 7f5ab4c0d740 20 datascan.operator(): loaded layout from xattr: pi: 5 os: 4194304 sc: 1 su: 4194304 2024-09-17T04:34:23.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.635+0000 7f5ab4c0d740 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-09-17T04:34:23.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.635+0000 7f5ab4c0d740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/sixmegs 2024-09-17T04:34:23.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.635+0000 7f5ab4c0d740 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=sixmegs 2024-09-17T04:34:23.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.635+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:118 6.10 6:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x55c628c66c00 con 0x55c628c68d80 2024-09-17T04:34:23.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.635+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.0 v2:172.21.15.37:6808/2283587593 9 ==== osd_op_reply(118 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 164+0+0 (crc 0 0 0) 0x55c628938500 con 0x55c628c68d80 2024-09-17T04:34:23.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.635+0000 7f5ab4c0d740 4 datascan.get_frag_of: Unexpected error on '10000000000.00000000': (2) No such file or directory 2024-09-17T04:34:23.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.635+0000 7f5ab4c0d740 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/sixmegs, will insert to root 2024-09-17T04:34:23.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.635+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:119 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x55c628c67000 con 0x55c628c68d80 2024-09-17T04:34:23.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.636+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.0 v2:172.21.15.37:6808/2283587593 10 ==== osd_op_reply(119 10000000000.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 164+0+0 (crc 0 0 0) 0x55c628938500 con 0x55c628c68d80 2024-09-17T04:34:23.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.636+0000 7f5ab4c0d740 10 datascan.find_or_create_dirfrag: read_version = 0 2024-09-17T04:34:23.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.636+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:120 6.10 6:0cf6ad0f:::10000000000.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x55c62978f400 con 0x55c628c68d80 2024-09-17T04:34:23.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.637+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.0 v2:172.21.15.37:6808/2283587593 11 ==== osd_op_reply(120 10000000000.00000000 [create,omap-set-header] v41'1 uv1 ondisk = 0) ==== 206+0+0 (crc 0 0 0) 0x55c628938500 con 0x55c628c68d80 2024-09-17T04:34:23.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.637+0000 7f5ab4c0d740 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x10000000000 2024-09-17T04:34:23.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.637+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:121 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x55c628c67400 con 0x55c628c68d80 2024-09-17T04:34:23.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.637+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.0 v2:172.21.15.37:6808/2283587593 12 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv1 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x55c628938500 con 0x55c628c68d80 2024-09-17T04:34:23.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.637+0000 7f5ab4c0d740 20 datascan.read_dentry: oid=10000000000.00000000 dname=sixmegs frag=*, r=0 2024-09-17T04:34:23.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.637+0000 7f5ab4c0d740 20 datascan.read_dentry: sixmegs_head not found in result 2024-09-17T04:34:23.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.637+0000 7f5ab4c0d740 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/sixmegs with size=6291456 bytes 2024-09-17T04:34:23.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.637+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:122 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=508b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x55c628c66800 con 0x55c628c68d80 2024-09-17T04:34:23.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.638+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.0 v2:172.21.15.37:6808/2283587593 13 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v41'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55c628938500 con 0x55c628c68d80 2024-09-17T04:34:23.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.638+0000 7f5ab4c0d740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/sixmegs pointing to 0x0x10000000001 2024-09-17T04:34:23.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.638+0000 7f5ab4c0d740 10 datascan.inject_with_backtrace: backptr: 0x0x1/subdir 2024-09-17T04:34:23.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.638+0000 7f5ab4c0d740 20 datascan.get_frag_of: dirino=0x1 target_dname=subdir 2024-09-17T04:34:23.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.638+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:123 6.1f 6:ff5b34d6:::1.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x55c628c66c00 con 0x55c628c52000 2024-09-17T04:34:23.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.639+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.3 v2:172.21.15.37:6800/809975826 20 ==== osd_op_reply(123 1.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 154+0+0 (crc 0 0 0) 0x55c628938500 con 0x55c628c52000 2024-09-17T04:34:23.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.639+0000 7f5ab4c0d740 4 datascan.get_frag_of: Unexpected error on '1.00000000': (2) No such file or directory 2024-09-17T04:34:23.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.639+0000 7f5ab4c0d740 20 datascan.inject_with_backtrace: don't know fragment for 0x0x1/subdir, will insert to root 2024-09-17T04:34:23.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.639+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:124 6.1f 6:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x55c62978f400 con 0x55c628c52000 2024-09-17T04:34:23.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.639+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.3 v2:172.21.15.37:6800/809975826 21 ==== osd_op_reply(124 1.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 154+0+0 (crc 0 0 0) 0x55c628938500 con 0x55c628c52000 2024-09-17T04:34:23.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.639+0000 7f5ab4c0d740 10 datascan.find_or_create_dirfrag: read_version = 0 2024-09-17T04:34:23.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.639+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:125 6.1f 6:ff5b34d6:::1.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x55c629729400 con 0x55c628c52000 2024-09-17T04:34:23.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.640+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.3 v2:172.21.15.37:6800/809975826 22 ==== osd_op_reply(125 1.00000000 [create,omap-set-header] v41'2 uv2 ondisk = 0) ==== 196+0+0 (crc 0 0 0) 0x55c628938500 con 0x55c628c52000 2024-09-17T04:34:23.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.640+0000 7f5ab4c0d740 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x1 2024-09-17T04:34:23.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.640+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:126 6.1f 6:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x55c628c70c00 con 0x55c628c52000 2024-09-17T04:34:23.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.641+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.3 v2:172.21.15.37:6800/809975826 23 ==== osd_op_reply(126 1.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv2 ondisk = 0) ==== 154+0+4 (crc 0 0 0) 0x55c628938500 con 0x55c628c52000 2024-09-17T04:34:23.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.641+0000 7f5ab4c0d740 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2024-09-17T04:34:23.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.641+0000 7f5ab4c0d740 20 datascan.read_dentry: subdir_head not found in result 2024-09-17T04:34:23.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.641+0000 7f5ab4c0d740 10 datascan.inject_with_backtrace: Linking ancestor directory of inode 0x0x10000000000 at 0x0x1:subdir 2024-09-17T04:34:23.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.641+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:127 6.1f 6:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x55c628c70400 con 0x55c628c52000 2024-09-17T04:34:23.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.642+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.3 v2:172.21.15.37:6800/809975826 24 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v41'3 uv3 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x55c628938500 con 0x55c628c52000 2024-09-17T04:34:23.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.642+0000 7f5ab4c0d740 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2024-09-17T04:34:23.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.642+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:128 5.27 5:e4000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c71800 con 0x55c628c68480 2024-09-17T04:34:23.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.642+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.1 v2:172.21.15.37:6824/2657955935 15 ==== osd_op_reply(128 [pgnls-filter start_epoch 41 out=49b] v33'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c68480 2024-09-17T04:34:23.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.642+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:129 5.17 5:e8000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c93400 con 0x55c628c52000 2024-09-17T04:34:23.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.642+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.3 v2:172.21.15.37:6800/809975826 25 ==== osd_op_reply(129 [pgnls-filter start_epoch 41 out=49b] v33'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938280 con 0x55c628c52000 2024-09-17T04:34:23.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.642+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:130 5.37 5:ec000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c92000 con 0x55c628c68000 2024-09-17T04:34:23.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.643+0000 7f5ab30b0640 1 -- 172.21.15.37:0/1380271642 <== osd.4 v2:172.21.15.92:6818/2509785752 15 ==== osd_op_reply(130 [pgnls-filter start_epoch 41 out=49b] v33'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938280 con 0x55c628c68000 2024-09-17T04:34:23.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.643+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:131 5.f 5:f0000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c93800 con 0x55c628c55600 2024-09-17T04:34:23.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.643+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.5 v2:172.21.15.92:6826/248263832 20 ==== osd_op_reply(131 [pgnls-filter start_epoch 41 out=49b] v33'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c628c55600 2024-09-17T04:34:23.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.643+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:132 5.2f 5:f4000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c92400 con 0x55c628c55180 2024-09-17T04:34:23.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.643+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 <== osd.7 v2:172.21.15.92:6802/3444787223 21 ==== osd_op_reply(132 [pgnls-filter start_epoch 41 out=49b] v33'9 uv1 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938780 con 0x55c628c55180 2024-09-17T04:34:23.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.643+0000 7f5aad8a5640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] -- osd_op(unknown.0.0:133 5.1f 5:f8000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c93c00 con 0x55c628c68900 2024-09-17T04:34:23.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.6 v2:172.21.15.92:6810/4078365985 7 ==== osd_op_reply(133 [pgnls-filter start_epoch 41 out=49b] v33'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c628c68900 2024-09-17T04:34:23.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5aae0a6640 1 -- 172.21.15.37:0/1380271642 --> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] -- osd_op(unknown.0.0:134 5.3f 5:fc000000::::head [pgnls-filter start_epoch 41 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55c628c92800 con 0x55c628c68900 2024-09-17T04:34:23.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab20ae640 1 -- 172.21.15.37:0/1380271642 <== osd.6 v2:172.21.15.92:6810/4078365985 8 ==== osd_op_reply(134 [pgnls-filter start_epoch 41 out=49b] v33'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c628938000 con 0x55c628c68900 2024-09-17T04:34:23.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x55c628c68d80 msgr2=0x55c628c64100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab4c0d740 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x55c628c68d80 0x55c628c64100 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x55c628c68480 msgr2=0x55c628c63600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab4c0d740 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x55c628c68480 0x55c628c63600 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x55c6289e1600 msgr2=0x55c629612680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab4c0d740 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x55c6289e1600 0x55c629612680 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x55c628c52000 msgr2=0x55c629612c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab4c0d740 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x55c628c52000 0x55c629612c00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x55c628c68000 msgr2=0x55c628c63080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab4c0d740 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x55c628c68000 0x55c628c63080 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x55c628c55600 msgr2=0x55c628c62580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab4c0d740 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x55c628c55600 0x55c628c62580 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 reap_dead start 2024-09-17T04:34:23.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] conn(0x55c628c68900 msgr2=0x55c628c63b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab4c0d740 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] conn(0x55c628c68900 0x55c628c63b80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab28af640 1 -- 172.21.15.37:0/1380271642 reap_dead start 2024-09-17T04:34:23.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x55c628c55180 msgr2=0x55c628c62000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab4c0d740 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x55c628c55180 0x55c628c62000 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x55c6289dff80 msgr2=0x55c629612100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab4c0d740 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x55c6289dff80 0x55c629612100 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55c62892b740 tx=0x55c62971b200 comp rx=0 tx=0).stop 2024-09-17T04:34:23.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6289e1180 msgr2=0x55c629611b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab4c0d740 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6289e1180 0x55c629611b80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55c629796630 tx=0x55c62971aea0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 shutdown_connections 2024-09-17T04:34:23.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab4c0d740 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x55c6289dff80 0x55c629612100 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab4c0d740 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6289e1180 0x55c629611b80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab4c0d740 1 --2- 172.21.15.37:0/1380271642 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x55c628c55180 0x55c628c62000 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.644+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 >> 172.21.15.37:0/1380271642 conn(0x55c6289de900 msgr2=0x55c628957000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.645+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 shutdown_connections 2024-09-17T04:34:23.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.645+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/1380271642 wait complete. 2024-09-17T04:34:23.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.645+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/2961998030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6289ded80 msgr2=0x55c629610b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.645+0000 7f5ab4c0d740 1 --2- 172.21.15.37:0/2961998030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6289ded80 0x55c629610b00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x55c62966fec0 tx=0x55c6288fef00 comp rx=0 tx=0).stop 2024-09-17T04:34:23.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.645+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/2961998030 shutdown_connections 2024-09-17T04:34:23.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.645+0000 7f5ab4c0d740 1 --2- 172.21.15.37:0/2961998030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6289ded80 0x55c629610b00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.645+0000 7f5ab4c0d740 1 --2- 172.21.15.37:0/2961998030 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55c6289df200 0x55c629611080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.645+0000 7f5ab4c0d740 1 --2- 172.21.15.37:0/2961998030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6289e1a80 0x55c629610580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.645+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/2961998030 >> 172.21.15.37:0/2961998030 conn(0x55c6289e0400 msgr2=0x55c628956800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.645+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/2961998030 shutdown_connections 2024-09-17T04:34:23.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.645+0000 7f5ab4c0d740 1 -- 172.21.15.37:0/2961998030 wait complete. 2024-09-17T04:34:23.654 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--alternate-pool', 'recovery_fs_metadata', '--filesystem', 'cephfs', '--force-corrupt', '--force-init', 'cephfs_data'] in time 0:00:00.134285, result: 2024-09-17T04:34:23.654 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links --filesystem recovery_fs 2024-09-17T04:34:23.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.707+0000 7ff3550bb740 1 -- start start 2024-09-17T04:34:23.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.707+0000 7ff3550bb740 1 Processor -- start 2024-09-17T04:34:23.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.708+0000 7ff3550bb740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55de981a3680 0x55de98dd8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.708+0000 7ff3550bb740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55de981a3b00 0x55de98dd8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.708+0000 7ff3550bb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de981a3f80 0x55de98dd8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.708+0000 7ff3550bb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55de98dfc180 con 0x55de981a3f80 2024-09-17T04:34:23.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.708+0000 7ff3550bb740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55de9805d800 con 0x55de981a3680 2024-09-17T04:34:23.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.708+0000 7ff3550bb740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x55de9805d380 con 0x55de981a3b00 2024-09-17T04:34:23.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.708+0000 7ff354063640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de981a3f80 0x55de98dd8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.709+0000 7ff354063640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de981a3f80 0x55de98dd8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38788/0 (socket says 172.21.15.37:38788) 2024-09-17T04:34:23.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.709+0000 7ff354063640 1 -- 172.21.15.37:0/736272864 learned_addr learned my addr 172.21.15.37:0/736272864 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.709+0000 7ff353061640 1 --2- 172.21.15.37:0/736272864 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55de981a3b00 0x55de98dd8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.709+0000 7ff353862640 1 --2- 172.21.15.37:0/736272864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55de981a3680 0x55de98dd8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.709+0000 7ff354063640 1 -- 172.21.15.37:0/736272864 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55de981a3b00 msgr2=0x55de98dd8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.709+0000 7ff354063640 1 --2- 172.21.15.37:0/736272864 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55de981a3b00 0x55de98dd8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.709+0000 7ff354063640 1 -- 172.21.15.37:0/736272864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55de981a3680 msgr2=0x55de98dd8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.709+0000 7ff354063640 1 --2- 172.21.15.37:0/736272864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55de981a3680 0x55de98dd8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.709+0000 7ff354063640 1 -- 172.21.15.37:0/736272864 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de93ff7680 con 0x55de981a3f80 2024-09-17T04:34:23.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.709+0000 7ff354063640 1 --2- 172.21.15.37:0/736272864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de981a3f80 0x55de98dd8000 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x55de98edc0c0 tx=0x55de980e8600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa939065f198391 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.709+0000 7ff35105d640 1 -- 172.21.15.37:0/736272864 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55de93ff0340 con 0x55de981a3f80 2024-09-17T04:34:23.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.709+0000 7ff35105d640 1 -- 172.21.15.37:0/736272864 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55de980508c0 con 0x55de981a3f80 2024-09-17T04:34:23.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.709+0000 7ff35105d640 1 -- 172.21.15.37:0/736272864 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55de93ff0ea0 con 0x55de981a3f80 2024-09-17T04:34:23.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.709+0000 7ff3550bb740 1 -- 172.21.15.37:0/736272864 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de93ff7860 con 0x55de981a3f80 2024-09-17T04:34:23.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.709+0000 7ff3550bb740 1 -- 172.21.15.37:0/736272864 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55de93ff72c0 con 0x55de981a3f80 2024-09-17T04:34:23.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.709+0000 7ff3550bb740 4 waiting for MDS map... 2024-09-17T04:34:23.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.709+0000 7ff35105d640 1 -- 172.21.15.37:0/736272864 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x55de98dfa000 con 0x55de981a3f80 2024-09-17T04:34:23.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.709+0000 7ff35105d640 1 -- 172.21.15.37:0/736272864 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 4143+0+0 (secure 0 0 0) 0x55de98055180 con 0x55de981a3f80 2024-09-17T04:34:23.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.709+0000 7ff3550bb740 4 Got MDS map 22 2024-09-17T04:34:23.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.710+0000 7ff3550bb740 4 datascan.main: Using metadata pool output 2024-09-17T04:34:23.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.710+0000 7ff3550bb740 4 datascan.main: connecting to RADOS... 2024-09-17T04:34:23.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.711+0000 7ff3550bb740 1 Processor -- start 2024-09-17T04:34:23.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.711+0000 7ff3550bb740 1 -- start start 2024-09-17T04:34:23.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.712+0000 7ff3550bb740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55de981a2d80 0x55de98dd9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.712+0000 7ff3550bb740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55de981a2900 0x55de98dd9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.712+0000 7ff3550bb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de981a4880 0x55de98dd9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.712+0000 7ff3550bb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55de9805d980 con 0x55de981a4880 2024-09-17T04:34:23.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.712+0000 7ff3550bb740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55de98dfd500 con 0x55de981a2d80 2024-09-17T04:34:23.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.712+0000 7ff3550bb740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x55de98dfd380 con 0x55de981a2900 2024-09-17T04:34:23.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.712+0000 7ff353061640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de981a4880 0x55de98dd9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.712+0000 7ff353061640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de981a4880 0x55de98dd9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38790/0 (socket says 172.21.15.37:38790) 2024-09-17T04:34:23.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.712+0000 7ff353061640 1 -- 172.21.15.37:0/2346228861 learned_addr learned my addr 172.21.15.37:0/2346228861 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.712+0000 7ff353862640 1 --2- 172.21.15.37:0/2346228861 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55de981a2900 0x55de98dd9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.712+0000 7ff353061640 1 --2- 172.21.15.37:0/2346228861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55de981a2d80 0x55de98dd9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.712+0000 7ff353061640 1 -- 172.21.15.37:0/2346228861 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55de981a2900 msgr2=0x55de98dd9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.712+0000 7ff353061640 1 --2- 172.21.15.37:0/2346228861 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55de981a2900 0x55de98dd9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.712+0000 7ff353061640 1 -- 172.21.15.37:0/2346228861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55de981a2d80 msgr2=0x55de98dd9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.712+0000 7ff353061640 1 --2- 172.21.15.37:0/2346228861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55de981a2d80 0x55de98dd9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.712+0000 7ff353061640 1 -- 172.21.15.37:0/2346228861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de93ff74a0 con 0x55de981a4880 2024-09-17T04:34:23.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.712+0000 7ff353061640 1 --2- 172.21.15.37:0/2346228861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de981a4880 0x55de98dd9b80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x55de98edd740 tx=0x55de98f56000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e665175d54311658 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.712+0000 7ff34f059640 1 -- 172.21.15.37:0/2346228861 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55de93ff0340 con 0x55de981a4880 2024-09-17T04:34:23.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.712+0000 7ff34f059640 1 -- 172.21.15.37:0/2346228861 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55de98050e00 con 0x55de981a4880 2024-09-17T04:34:23.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.712+0000 7ff34f059640 1 -- 172.21.15.37:0/2346228861 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55de93ff0ea0 con 0x55de981a4880 2024-09-17T04:34:23.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.713+0000 7ff3550bb740 1 -- 172.21.15.37:0/2346228861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de981a4880 msgr2=0x55de98dd9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.713+0000 7ff3550bb740 1 --2- 172.21.15.37:0/2346228861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de981a4880 0x55de98dd9b80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x55de98edd740 tx=0x55de98f56000 comp rx=0 tx=0).stop 2024-09-17T04:34:23.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.713+0000 7ff3550bb740 1 -- 172.21.15.37:0/2346228861 shutdown_connections 2024-09-17T04:34:23.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.713+0000 7ff3550bb740 1 --2- 172.21.15.37:0/2346228861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de981a4880 0x55de98dd9b80 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.713+0000 7ff3550bb740 1 --2- 172.21.15.37:0/2346228861 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55de981a2900 0x55de98dd9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.713+0000 7ff3550bb740 1 --2- 172.21.15.37:0/2346228861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55de981a2d80 0x55de98dd9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.713+0000 7ff3550bb740 1 -- 172.21.15.37:0/2346228861 >> 172.21.15.37:0/2346228861 conn(0x55de981a3200 msgr2=0x55de9811d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.713+0000 7ff3550bb740 1 -- 172.21.15.37:0/2346228861 shutdown_connections 2024-09-17T04:34:23.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.713+0000 7ff3550bb740 1 -- 172.21.15.37:0/2346228861 wait complete. 2024-09-17T04:34:23.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.714+0000 7ff3550bb740 1 Processor -- start 2024-09-17T04:34:23.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.714+0000 7ff3550bb740 1 -- start start 2024-09-17T04:34:23.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.715+0000 7ff3550bb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de981a2900 0x55de98dd9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.715+0000 7ff3550bb740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55de981a2d80 0x55de98dd9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.715+0000 7ff3550bb740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55de981a4880 0x55de98dd9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.715+0000 7ff3550bb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55de98dfd680 con 0x55de981a2900 2024-09-17T04:34:23.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.715+0000 7ff3550bb740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55de98dfc900 con 0x55de981a2d80 2024-09-17T04:34:23.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.715+0000 7ff3550bb740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x55de9805d380 con 0x55de981a4880 2024-09-17T04:34:23.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.715+0000 7ff353061640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de981a2900 0x55de98dd9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.715+0000 7ff353061640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de981a2900 0x55de98dd9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38792/0 (socket says 172.21.15.37:38792) 2024-09-17T04:34:23.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.715+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 learned_addr learned my addr 172.21.15.37:0/4086956892 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.715+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55de981a4880 msgr2=0x55de98dd9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.715+0000 7ff353061640 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55de981a4880 0x55de98dd9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.715+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55de981a2d80 msgr2=0x55de98dd9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.715+0000 7ff353061640 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55de981a2d80 0x55de98dd9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.715+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de93ff70e0 con 0x55de981a2900 2024-09-17T04:34:23.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.716+0000 7ff353061640 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de981a2900 0x55de98dd9600 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x55de98f446c0 tx=0x55de98f561e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.716+0000 7ff34d856640 1 -- 172.21.15.37:0/4086956892 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55de93ff1040 con 0x55de981a2900 2024-09-17T04:34:23.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.716+0000 7ff34d856640 1 -- 172.21.15.37:0/4086956892 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55de98050540 con 0x55de981a2900 2024-09-17T04:34:23.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.716+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55de93ff7a40 con 0x55de981a2900 2024-09-17T04:34:23.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.716+0000 7ff34d856640 1 -- 172.21.15.37:0/4086956892 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55de93ff0340 con 0x55de981a2900 2024-09-17T04:34:23.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.716+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de93ff63c0 con 0x55de981a2900 2024-09-17T04:34:23.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.716+0000 7ff3550bb740 4 datascan.init: resolving metadata pool 6 2024-09-17T04:34:23.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.717+0000 7ff34d856640 1 -- 172.21.15.37:0/4086956892 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x55de98f38900 con 0x55de981a2900 2024-09-17T04:34:23.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.718+0000 7ff34d856640 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x55de981a5a80 0x55de98dda100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.718+0000 7ff34d856640 1 -- 172.21.15.37:0/4086956892 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x55de98f42a00 con 0x55de981a2900 2024-09-17T04:34:23.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.718+0000 7ff3550bb740 4 datascan.init: found metadata pool 'recovery_fs_metadata' 2024-09-17T04:34:23.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.718+0000 7ff3550bb740 4 datascan.main: resolving metadata pool 6 2024-09-17T04:34:23.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.718+0000 7ff353862640 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x55de981a5a80 0x55de98dda100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.718+0000 7ff3550bb740 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x55de981a4d00 0x55de98dda680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.718+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:1 6.0 6:00000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de981a4d00 2024-09-17T04:34:23.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.718+0000 7ff353061640 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x55de981a4d00 0x55de98dda680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.718+0000 7ff353862640 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x55de981a5a80 0x55de98dda100 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55de980ef590 tx=0x55de98f56600 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.718+0000 7ff353061640 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x55de981a4d00 0x55de98dda680 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.719+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 <== osd.0 v2:172.21.15.37:6808/2283587593 1 ==== osd_op_reply(1 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa000 con 0x55de981a4d00 2024-09-17T04:34:23.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.719+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:2 6.10 6:08000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de981a4d00 2024-09-17T04:34:23.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.719+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 <== osd.0 v2:172.21.15.37:6808/2283587593 2 ==== osd_op_reply(2 [pgnls start_epoch 41 out=81b] v41'2 uv2 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55de980fa000 con 0x55de981a4d00 2024-09-17T04:34:23.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.719+0000 7ff353061640 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x55de98414000 0x55de98418000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.719+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:3 6.8 6:10000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98414000 2024-09-17T04:34:23.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.719+0000 7ff354063640 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x55de98414000 0x55de98418000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.719+0000 7ff354063640 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x55de98414000 0x55de98418000 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.720+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.7 v2:172.21.15.92:6802/3444787223 1 ==== osd_op_reply(3 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa500 con 0x55de98414000 2024-09-17T04:34:23.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.720+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:4 6.18 6:18000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de981a4d00 2024-09-17T04:34:23.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.720+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 <== osd.0 v2:172.21.15.37:6808/2283587593 3 ==== osd_op_reply(4 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa000 con 0x55de981a4d00 2024-09-17T04:34:23.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.720+0000 7ff353061640 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x55de98414480 0x55de98418580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.720+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:5 6.4 6:20000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98414480 2024-09-17T04:34:23.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.720+0000 7ff353862640 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x55de98414480 0x55de98418580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.721+0000 7ff353862640 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x55de98414480 0x55de98418580 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.721+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 <== osd.1 v2:172.21.15.37:6824/2657955935 1 ==== osd_op_reply(5 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa780 con 0x55de98414480 2024-09-17T04:34:23.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.721+0000 7ff353862640 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x55de98414d80 0x55de98419080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.721+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:6 6.14 6:28000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98414d80 2024-09-17T04:34:23.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.721+0000 7ff353061640 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x55de98414d80 0x55de98419080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.721+0000 7ff353061640 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x55de98414d80 0x55de98419080 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.722+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 <== osd.2 v2:172.21.15.37:6816/579720176 1 ==== osd_op_reply(6 [pgnls start_epoch 41 out=77b] v41'1 uv1 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x55de980fa000 con 0x55de98414d80 2024-09-17T04:34:23.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.722+0000 7ff353061640 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x55de98414900 0x55de98418b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.722+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:7 6.c 6:30000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98414900 2024-09-17T04:34:23.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.722+0000 7ff354063640 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x55de98414900 0x55de98418b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.722+0000 7ff354063640 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x55de98414900 0x55de98418b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.722+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.3 v2:172.21.15.37:6800/809975826 1 ==== osd_op_reply(7 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa500 con 0x55de98414900 2024-09-17T04:34:23.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.722+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:8 6.1c 6:38000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98414000 2024-09-17T04:34:23.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.723+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.7 v2:172.21.15.92:6802/3444787223 2 ==== osd_op_reply(8 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa500 con 0x55de98414000 2024-09-17T04:34:23.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.723+0000 7ff354063640 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x55de98415200 0x55de98419600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.723+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:9 6.2 6:40000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98415200 2024-09-17T04:34:23.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.723+0000 7ff353862640 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x55de98415200 0x55de98419600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.723+0000 7ff353862640 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x55de98415200 0x55de98419600 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.724+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 <== osd.4 v2:172.21.15.92:6818/2509785752 1 ==== osd_op_reply(9 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa780 con 0x55de98415200 2024-09-17T04:34:23.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.724+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:10 6.12 6:48000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98414000 2024-09-17T04:34:23.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.724+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.7 v2:172.21.15.92:6802/3444787223 3 ==== osd_op_reply(10 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa500 con 0x55de98414000 2024-09-17T04:34:23.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.724+0000 7ff354063640 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x55de98415680 0x55de98419b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.724+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:11 6.a 6:50000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98415680 2024-09-17T04:34:23.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.724+0000 7ff353061640 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x55de98415680 0x55de98419b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.725+0000 7ff353061640 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x55de98415680 0x55de98419b80 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.725+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 <== osd.5 v2:172.21.15.92:6826/248263832 1 ==== osd_op_reply(11 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa000 con 0x55de98415680 2024-09-17T04:34:23.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.725+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:12 6.1a 6:58000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98415200 2024-09-17T04:34:23.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.725+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 <== osd.4 v2:172.21.15.92:6818/2509785752 2 ==== osd_op_reply(12 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa780 con 0x55de98415200 2024-09-17T04:34:23.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.725+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:13 6.6 6:60000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98414900 2024-09-17T04:34:23.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.725+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.3 v2:172.21.15.37:6800/809975826 2 ==== osd_op_reply(13 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa500 con 0x55de98414900 2024-09-17T04:34:23.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.726+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:14 6.16 6:68000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de981a4d00 2024-09-17T04:34:23.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.726+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 <== osd.0 v2:172.21.15.37:6808/2283587593 4 ==== osd_op_reply(14 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa000 con 0x55de981a4d00 2024-09-17T04:34:23.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.726+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:15 6.e 6:70000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98415200 2024-09-17T04:34:23.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.726+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 <== osd.4 v2:172.21.15.92:6818/2509785752 3 ==== osd_op_reply(15 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa780 con 0x55de98415200 2024-09-17T04:34:23.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.726+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:16 6.1e 6:78000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98415200 2024-09-17T04:34:23.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.726+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 <== osd.4 v2:172.21.15.92:6818/2509785752 4 ==== osd_op_reply(16 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa780 con 0x55de98415200 2024-09-17T04:34:23.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.726+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:17 6.1 6:80000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98414480 2024-09-17T04:34:23.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.727+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 <== osd.1 v2:172.21.15.37:6824/2657955935 2 ==== osd_op_reply(17 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa780 con 0x55de98414480 2024-09-17T04:34:23.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.727+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:18 6.11 6:88000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98414900 2024-09-17T04:34:23.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.727+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.3 v2:172.21.15.37:6800/809975826 3 ==== osd_op_reply(18 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa500 con 0x55de98414900 2024-09-17T04:34:23.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.727+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:19 6.9 6:90000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de981a4d00 2024-09-17T04:34:23.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.727+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 <== osd.0 v2:172.21.15.37:6808/2283587593 5 ==== osd_op_reply(19 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa000 con 0x55de981a4d00 2024-09-17T04:34:23.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.727+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:20 6.19 6:98000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98415680 2024-09-17T04:34:23.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.728+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 <== osd.5 v2:172.21.15.92:6826/248263832 2 ==== osd_op_reply(20 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa000 con 0x55de98415680 2024-09-17T04:34:23.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.728+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:21 6.5 6:a0000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98414000 2024-09-17T04:34:23.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.728+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.7 v2:172.21.15.92:6802/3444787223 4 ==== osd_op_reply(21 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa500 con 0x55de98414000 2024-09-17T04:34:23.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.728+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:22 6.15 6:a8000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98414000 2024-09-17T04:34:23.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.728+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.7 v2:172.21.15.92:6802/3444787223 5 ==== osd_op_reply(22 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa500 con 0x55de98414000 2024-09-17T04:34:23.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.728+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:23 6.d 6:b0000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98415680 2024-09-17T04:34:23.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.729+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 <== osd.5 v2:172.21.15.92:6826/248263832 3 ==== osd_op_reply(23 [pgnls start_epoch 41 out=74b] v41'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x55de980fa000 con 0x55de98415680 2024-09-17T04:34:23.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.729+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:24 6.1d 6:b8000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98414480 2024-09-17T04:34:23.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.729+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 <== osd.1 v2:172.21.15.37:6824/2657955935 3 ==== osd_op_reply(24 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa780 con 0x55de98414480 2024-09-17T04:34:23.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.729+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:25 6.3 6:c0000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98414000 2024-09-17T04:34:23.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.729+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.7 v2:172.21.15.92:6802/3444787223 6 ==== osd_op_reply(25 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa500 con 0x55de98414000 2024-09-17T04:34:23.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.729+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:26 6.13 6:c8000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98414900 2024-09-17T04:34:23.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.730+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.3 v2:172.21.15.37:6800/809975826 4 ==== osd_op_reply(26 [pgnls start_epoch 41 out=98b] v41'2 uv2 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x55de980fa500 con 0x55de98414900 2024-09-17T04:34:23.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.730+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:27 6.b 6:d0000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98414900 2024-09-17T04:34:23.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.730+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.3 v2:172.21.15.37:6800/809975826 5 ==== osd_op_reply(27 [pgnls start_epoch 41 out=76b] v41'1 uv1 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x55de980fa500 con 0x55de98414900 2024-09-17T04:34:23.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.730+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:28 6.1b 6:d8000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98414900 2024-09-17T04:34:23.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.730+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.3 v2:172.21.15.37:6800/809975826 6 ==== osd_op_reply(28 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa500 con 0x55de98414900 2024-09-17T04:34:23.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.730+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:29 6.7 6:e0000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98415680 2024-09-17T04:34:23.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.730+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 <== osd.5 v2:172.21.15.92:6826/248263832 4 ==== osd_op_reply(29 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa000 con 0x55de98415680 2024-09-17T04:34:23.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.730+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:30 6.17 6:e8000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98415200 2024-09-17T04:34:23.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.731+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 <== osd.4 v2:172.21.15.92:6818/2509785752 5 ==== osd_op_reply(30 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa780 con 0x55de98415200 2024-09-17T04:34:23.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.731+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:31 6.f 6:f0000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98414d80 2024-09-17T04:34:23.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.731+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 <== osd.2 v2:172.21.15.37:6816/579720176 2 ==== osd_op_reply(31 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa000 con 0x55de98414d80 2024-09-17T04:34:23.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.731+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:32 6.1f 6:f8000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98414900 2024-09-17T04:34:23.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.731+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.3 v2:172.21.15.37:6800/809975826 7 ==== osd_op_reply(32 [pgnls start_epoch 41 out=101b] v41'3 uv3 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x55de980fa500 con 0x55de98414900 2024-09-17T04:34:23.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.731+0000 7ff3550bb740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-09-17T04:34:23.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.731+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:33 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58c00 con 0x55de981a4d00 2024-09-17T04:34:23.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.732+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 <== osd.0 v2:172.21.15.37:6808/2283587593 6 ==== osd_op_reply(33 10000000000.00000000 [omap-get-vals out=509b] v0'0 uv2 ondisk = 0) ==== 164+0+509 (crc 0 0 0) 0x55de980fa000 con 0x55de981a4d00 2024-09-17T04:34:23.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.732+0000 7ff3550bb740 20 datascan.scan_links: injected ino detected 2024-09-17T04:34:23.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.732+0000 7ff3550bb740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-09-17T04:34:23.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.732+0000 7ff3550bb740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-09-17T04:34:23.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.732+0000 7ff3550bb740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-09-17T04:34:23.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.732+0000 7ff3550bb740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-09-17T04:34:23.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.732+0000 7ff3550bb740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-09-17T04:34:23.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.732+0000 7ff3550bb740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-09-17T04:34:23.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.732+0000 7ff3550bb740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-09-17T04:34:23.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.732+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:34 6.13 6:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x55de98f59c00 con 0x55de98414900 2024-09-17T04:34:23.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.732+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.3 v2:172.21.15.37:6800/809975826 8 ==== osd_op_reply(34 100.00000000 [omap-get-vals out=5b] v0'0 uv1 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55de980fa500 con 0x55de98414900 2024-09-17T04:34:23.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.732+0000 7ff3550bb740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-09-17T04:34:23.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.732+0000 7ff3550bb740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-09-17T04:34:23.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.732+0000 7ff3550bb740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-09-17T04:34:23.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.732+0000 7ff3550bb740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-09-17T04:34:23.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.732+0000 7ff3550bb740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-09-17T04:34:23.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.732+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:35 6.1f 6:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a000 con 0x55de98414900 2024-09-17T04:34:23.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.732+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.3 v2:172.21.15.37:6800/809975826 9 ==== osd_op_reply(35 1.00000000 [omap-get-vals out=508b] v0'0 uv3 ondisk = 0) ==== 154+0+508 (crc 0 0 0) 0x55de980fa500 con 0x55de98414900 2024-09-17T04:34:23.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.733+0000 7ff3550bb740 20 datascan.scan_links: injected ino detected 2024-09-17T04:34:23.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.733+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:36 6.0 6:00000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de981a4d00 2024-09-17T04:34:23.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.733+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 <== osd.0 v2:172.21.15.37:6808/2283587593 7 ==== osd_op_reply(36 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa000 con 0x55de981a4d00 2024-09-17T04:34:23.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.733+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:37 6.10 6:08000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de981a4d00 2024-09-17T04:34:23.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.733+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 <== osd.0 v2:172.21.15.37:6808/2283587593 8 ==== osd_op_reply(37 [pgnls start_epoch 41 out=81b] v41'2 uv2 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55de980fa000 con 0x55de981a4d00 2024-09-17T04:34:23.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.733+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:38 6.8 6:10000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98414000 2024-09-17T04:34:23.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.733+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.7 v2:172.21.15.92:6802/3444787223 7 ==== osd_op_reply(38 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa500 con 0x55de98414000 2024-09-17T04:34:23.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.733+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:39 6.18 6:18000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de981a4d00 2024-09-17T04:34:23.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.734+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 <== osd.0 v2:172.21.15.37:6808/2283587593 9 ==== osd_op_reply(39 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa000 con 0x55de981a4d00 2024-09-17T04:34:23.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.734+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:40 6.4 6:20000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98414480 2024-09-17T04:34:23.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.734+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 <== osd.1 v2:172.21.15.37:6824/2657955935 4 ==== osd_op_reply(40 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa780 con 0x55de98414480 2024-09-17T04:34:23.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.734+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:41 6.14 6:28000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98414d80 2024-09-17T04:34:23.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.734+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 <== osd.2 v2:172.21.15.37:6816/579720176 3 ==== osd_op_reply(41 [pgnls start_epoch 41 out=77b] v41'1 uv1 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x55de980fa000 con 0x55de98414d80 2024-09-17T04:34:23.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.734+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:42 6.c 6:30000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98414900 2024-09-17T04:34:23.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.735+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.3 v2:172.21.15.37:6800/809975826 10 ==== osd_op_reply(42 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa500 con 0x55de98414900 2024-09-17T04:34:23.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.735+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:43 6.1c 6:38000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98414000 2024-09-17T04:34:23.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.735+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.7 v2:172.21.15.92:6802/3444787223 8 ==== osd_op_reply(43 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa500 con 0x55de98414000 2024-09-17T04:34:23.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.735+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:44 6.2 6:40000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98415200 2024-09-17T04:34:23.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.735+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 <== osd.4 v2:172.21.15.92:6818/2509785752 6 ==== osd_op_reply(44 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa780 con 0x55de98415200 2024-09-17T04:34:23.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.735+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:45 6.12 6:48000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98414000 2024-09-17T04:34:23.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.736+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.7 v2:172.21.15.92:6802/3444787223 9 ==== osd_op_reply(45 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa500 con 0x55de98414000 2024-09-17T04:34:23.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.736+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:46 6.a 6:50000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98415680 2024-09-17T04:34:23.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.736+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 <== osd.5 v2:172.21.15.92:6826/248263832 5 ==== osd_op_reply(46 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa000 con 0x55de98415680 2024-09-17T04:34:23.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.736+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:47 6.1a 6:58000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98415200 2024-09-17T04:34:23.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.736+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 <== osd.4 v2:172.21.15.92:6818/2509785752 7 ==== osd_op_reply(47 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa780 con 0x55de98415200 2024-09-17T04:34:23.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.736+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:48 6.6 6:60000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98414900 2024-09-17T04:34:23.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.736+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.3 v2:172.21.15.37:6800/809975826 11 ==== osd_op_reply(48 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa500 con 0x55de98414900 2024-09-17T04:34:23.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.737+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:49 6.16 6:68000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de981a4d00 2024-09-17T04:34:23.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.737+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 <== osd.0 v2:172.21.15.37:6808/2283587593 10 ==== osd_op_reply(49 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa000 con 0x55de981a4d00 2024-09-17T04:34:23.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.737+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:50 6.e 6:70000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98415200 2024-09-17T04:34:23.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.737+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 <== osd.4 v2:172.21.15.92:6818/2509785752 8 ==== osd_op_reply(50 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa780 con 0x55de98415200 2024-09-17T04:34:23.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.737+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:51 6.1e 6:78000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98415200 2024-09-17T04:34:23.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.737+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 <== osd.4 v2:172.21.15.92:6818/2509785752 9 ==== osd_op_reply(51 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa780 con 0x55de98415200 2024-09-17T04:34:23.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.737+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:52 6.1 6:80000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98414480 2024-09-17T04:34:23.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.738+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 <== osd.1 v2:172.21.15.37:6824/2657955935 5 ==== osd_op_reply(52 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa780 con 0x55de98414480 2024-09-17T04:34:23.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.738+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:53 6.11 6:88000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98414900 2024-09-17T04:34:23.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.738+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.3 v2:172.21.15.37:6800/809975826 12 ==== osd_op_reply(53 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa500 con 0x55de98414900 2024-09-17T04:34:23.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.738+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:54 6.9 6:90000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de981a4d00 2024-09-17T04:34:23.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.738+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 <== osd.0 v2:172.21.15.37:6808/2283587593 11 ==== osd_op_reply(54 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa000 con 0x55de981a4d00 2024-09-17T04:34:23.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.738+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:55 6.19 6:98000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98415680 2024-09-17T04:34:23.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.739+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 <== osd.5 v2:172.21.15.92:6826/248263832 6 ==== osd_op_reply(55 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa000 con 0x55de98415680 2024-09-17T04:34:23.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.739+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:56 6.5 6:a0000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98414000 2024-09-17T04:34:23.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.739+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.7 v2:172.21.15.92:6802/3444787223 10 ==== osd_op_reply(56 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa500 con 0x55de98414000 2024-09-17T04:34:23.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.739+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:57 6.15 6:a8000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98414000 2024-09-17T04:34:23.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.739+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.7 v2:172.21.15.92:6802/3444787223 11 ==== osd_op_reply(57 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa500 con 0x55de98414000 2024-09-17T04:34:23.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.739+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:58 6.d 6:b0000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98415680 2024-09-17T04:34:23.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.739+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 <== osd.5 v2:172.21.15.92:6826/248263832 7 ==== osd_op_reply(58 [pgnls start_epoch 41 out=74b] v41'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x55de980fa000 con 0x55de98415680 2024-09-17T04:34:23.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.739+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- osd_op(unknown.0.0:59 6.1d 6:b8000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98414480 2024-09-17T04:34:23.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.740+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 <== osd.1 v2:172.21.15.37:6824/2657955935 6 ==== osd_op_reply(59 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa780 con 0x55de98414480 2024-09-17T04:34:23.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.740+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:60 6.3 6:c0000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98414000 2024-09-17T04:34:23.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.740+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.7 v2:172.21.15.92:6802/3444787223 12 ==== osd_op_reply(60 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa500 con 0x55de98414000 2024-09-17T04:34:23.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.740+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:61 6.13 6:c8000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98414900 2024-09-17T04:34:23.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.740+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.3 v2:172.21.15.37:6800/809975826 13 ==== osd_op_reply(61 [pgnls start_epoch 41 out=98b] v41'2 uv2 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x55de980fa500 con 0x55de98414900 2024-09-17T04:34:23.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.740+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:62 6.b 6:d0000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98414900 2024-09-17T04:34:23.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.741+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.3 v2:172.21.15.37:6800/809975826 14 ==== osd_op_reply(62 [pgnls start_epoch 41 out=76b] v41'1 uv1 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x55de980fa500 con 0x55de98414900 2024-09-17T04:34:23.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.741+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:63 6.1b 6:d8000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98414900 2024-09-17T04:34:23.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.741+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.3 v2:172.21.15.37:6800/809975826 15 ==== osd_op_reply(63 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa500 con 0x55de98414900 2024-09-17T04:34:23.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.741+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:64 6.7 6:e0000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98415680 2024-09-17T04:34:23.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.741+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 <== osd.5 v2:172.21.15.92:6826/248263832 8 ==== osd_op_reply(64 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa000 con 0x55de98415680 2024-09-17T04:34:23.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.741+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:65 6.17 6:e8000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98415200 2024-09-17T04:34:23.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.742+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 <== osd.4 v2:172.21.15.92:6818/2509785752 10 ==== osd_op_reply(65 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa780 con 0x55de98415200 2024-09-17T04:34:23.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.742+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:66 6.f 6:f0000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98414d80 2024-09-17T04:34:23.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.742+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 <== osd.2 v2:172.21.15.37:6816/579720176 4 ==== osd_op_reply(66 [pgnls start_epoch 41 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55de980fa000 con 0x55de98414d80 2024-09-17T04:34:23.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.742+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:67 6.1f 6:f8000000::::head [pgnls start_epoch 41 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98414900 2024-09-17T04:34:23.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.742+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.3 v2:172.21.15.37:6800/809975826 16 ==== osd_op_reply(67 [pgnls start_epoch 41 out=101b] v41'3 uv3 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x55de980fa500 con 0x55de98414900 2024-09-17T04:34:23.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.742+0000 7ff3550bb740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-09-17T04:34:23.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.742+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:68 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a800 con 0x55de981a4d00 2024-09-17T04:34:23.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 <== osd.0 v2:172.21.15.37:6808/2283587593 12 ==== osd_op_reply(68 10000000000.00000000 [omap-get-vals out=509b] v0'0 uv2 ondisk = 0) ==== 164+0+509 (crc 0 0 0) 0x55de980fa000 con 0x55de981a4d00 2024-09-17T04:34:23.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff3550bb740 20 datascan.scan_links: injected ino detected 2024-09-17T04:34:23.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff3550bb740 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-09-17T04:34:23.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff3550bb740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-09-17T04:34:23.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff3550bb740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-09-17T04:34:23.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff3550bb740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-09-17T04:34:23.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff3550bb740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-09-17T04:34:23.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff3550bb740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-09-17T04:34:23.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff3550bb740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-09-17T04:34:23.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff3550bb740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-09-17T04:34:23.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:69 6.13 6:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x55de9843ac00 con 0x55de98414900 2024-09-17T04:34:23.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.3 v2:172.21.15.37:6800/809975826 17 ==== osd_op_reply(69 100.00000000 [omap-get-vals out=5b] v0'0 uv1 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55de980fa500 con 0x55de98414900 2024-09-17T04:34:23.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff3550bb740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-09-17T04:34:23.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff3550bb740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-09-17T04:34:23.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff3550bb740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-09-17T04:34:23.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff3550bb740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-09-17T04:34:23.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff3550bb740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-09-17T04:34:23.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:70 6.1f 6:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a000 con 0x55de98414900 2024-09-17T04:34:23.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.3 v2:172.21.15.37:6800/809975826 18 ==== osd_op_reply(70 1.00000000 [omap-get-vals out=508b] v0'0 uv3 ondisk = 0) ==== 154+0+508 (crc 0 0 0) 0x55de980fa500 con 0x55de98414900 2024-09-17T04:34:23.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff3550bb740 20 datascan.scan_links: injected ino detected 2024-09-17T04:34:23.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff3550bb740 20 datascan.scan_links: adding 0x10000000000 for future processing to fix dnfirst 2024-09-17T04:34:23.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff3550bb740 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-09-17T04:34:23.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff3550bb740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-09-17T04:34:23.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff3550bb740 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-09-17T04:34:23.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff3550bb740 10 datascan.scan_links: processing 2 injected_inos 2024-09-17T04:34:23.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff3550bb740 10 datascan.scan_links: handling injected_ino 0x10000000000 2024-09-17T04:34:23.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff3550bb740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x1:subdir 2024-09-17T04:34:23.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.743+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:71 6.1f 6:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a400 con 0x55de98414900 2024-09-17T04:34:23.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.744+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.3 v2:172.21.15.37:6800/809975826 19 ==== osd_op_reply(71 1.00000000 [omap-get-vals-by-keys out=507b] v0'0 uv3 ondisk = 0) ==== 154+0+507 (crc 0 0 0) 0x55de980fa500 con 0x55de98414900 2024-09-17T04:34:23.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.744+0000 7ff3550bb740 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2024-09-17T04:34:23.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.744+0000 7ff3550bb740 20 datascan.scan_links: first is now 2 2024-09-17T04:34:23.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.744+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:72 6.1f 6:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x55de98f58800 con 0x55de98414900 2024-09-17T04:34:23.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.745+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.3 v2:172.21.15.37:6800/809975826 20 ==== osd_op_reply(72 1.00000000 [omap-set-vals] v41'4 uv4 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x55de980fa500 con 0x55de98414900 2024-09-17T04:34:23.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.745+0000 7ff3550bb740 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2024-09-17T04:34:23.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.745+0000 7ff3550bb740 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-09-17T04:34:23.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.745+0000 7ff3550bb740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:sixmegs 2024-09-17T04:34:23.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.745+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:73 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x55de98f59c00 con 0x55de981a4d00 2024-09-17T04:34:23.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.745+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 <== osd.0 v2:172.21.15.37:6808/2283587593 13 ==== osd_op_reply(73 10000000000.00000000 [omap-get-vals-by-keys out=508b] v0'0 uv2 ondisk = 0) ==== 164+0+508 (crc 0 0 0) 0x55de980fa000 con 0x55de981a4d00 2024-09-17T04:34:23.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.745+0000 7ff3550bb740 20 datascan.read_dentry: oid=10000000000.00000000 dname=sixmegs frag=*, r=0 2024-09-17T04:34:23.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.745+0000 7ff3550bb740 20 datascan.scan_links: first is now 2 2024-09-17T04:34:23.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.745+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- osd_op(unknown.0.0:74 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=508b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x55de98df8c00 con 0x55de981a4d00 2024-09-17T04:34:23.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.746+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 <== osd.0 v2:172.21.15.37:6808/2283587593 14 ==== osd_op_reply(74 10000000000.00000000 [omap-set-vals] v41'3 uv3 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55de980fa000 con 0x55de981a4d00 2024-09-17T04:34:23.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.746+0000 7ff3550bb740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/sixmegs pointing to 0x0x10000000001 2024-09-17T04:34:23.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.746+0000 7ff3550bb740 10 datascan.scan_links: updating inotable 2024-09-17T04:34:23.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.746+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:75 6.13 6:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x55de9843a800 con 0x55de98414900 2024-09-17T04:34:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.747+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.3 v2:172.21.15.37:6800/809975826 21 ==== osd_op_reply(75 mds0_inotable [read 0~34 out=34b] v0'0 uv2 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x55de980fa500 con 0x55de98414900 2024-09-17T04:34:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.747+0000 7ff3550bb740 10 mds.0.inotable: skip_inos was [0x10000000000~0x10000000000] 2024-09-17T04:34:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.747+0000 7ff3550bb740 10 mds.0.inotable: skip_inos now [0x10000000002~0xfffffffffe] 2024-09-17T04:34:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.747+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:76 6.13 6:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x55de9843ac00 con 0x55de98414900 2024-09-17T04:34:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.749+0000 7ff354063640 1 -- 172.21.15.37:0/4086956892 <== osd.3 v2:172.21.15.37:6800/809975826 22 ==== osd_op_reply(76 mds0_inotable [writefull 0~34] v41'3 uv3 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x55de980fa500 con 0x55de98414900 2024-09-17T04:34:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.749+0000 7ff3550bb740 10 datascan.scan_links: updating snaptable 2024-09-17T04:34:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.749+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- osd_op(unknown.0.0:77 6.d 6:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x55de9843b000 con 0x55de98415680 2024-09-17T04:34:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff353061640 1 -- 172.21.15.37:0/4086956892 <== osd.5 v2:172.21.15.92:6826/248263832 9 ==== osd_op_reply(77 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x55de980fa000 con 0x55de98415680 2024-09-17T04:34:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x55de981a4d00 msgr2=0x55de98dda680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff3550bb740 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x55de981a4d00 0x55de98dda680 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x55de98414480 msgr2=0x55de98418580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff3550bb740 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x55de98414480 0x55de98418580 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x55de98414d80 msgr2=0x55de98419080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff3550bb740 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x55de98414d80 0x55de98419080 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x55de98414900 msgr2=0x55de98418b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff3550bb740 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x55de98414900 0x55de98418b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x55de98415200 msgr2=0x55de98419600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff3550bb740 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x55de98415200 0x55de98419600 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x55de98415680 msgr2=0x55de98419b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff3550bb740 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x55de98415680 0x55de98419b80 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 reap_dead start 2024-09-17T04:34:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x55de98414000 msgr2=0x55de98418000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff3550bb740 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x55de98414000 0x55de98418000 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 reap_dead start 2024-09-17T04:34:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff353862640 1 -- 172.21.15.37:0/4086956892 reap_dead start 2024-09-17T04:34:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x55de981a5a80 msgr2=0x55de98dda100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff3550bb740 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x55de981a5a80 0x55de98dda100 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55de980ef590 tx=0x55de98f56600 comp rx=0 tx=0).stop 2024-09-17T04:34:23.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de981a2900 msgr2=0x55de98dd9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff3550bb740 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de981a2900 0x55de98dd9600 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x55de98f446c0 tx=0x55de98f561e0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 shutdown_connections 2024-09-17T04:34:23.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff3550bb740 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x55de981a5a80 0x55de98dda100 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff3550bb740 1 --2- 172.21.15.37:0/4086956892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de981a2900 0x55de98dd9600 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 >> 172.21.15.37:0/4086956892 conn(0x55de981a3200 msgr2=0x55de9811d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 shutdown_connections 2024-09-17T04:34:23.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.750+0000 7ff3550bb740 1 -- 172.21.15.37:0/4086956892 wait complete. 2024-09-17T04:34:23.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.751+0000 7ff3550bb740 1 -- 172.21.15.37:0/736272864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de981a3f80 msgr2=0x55de98dd8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.751+0000 7ff3550bb740 1 --2- 172.21.15.37:0/736272864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de981a3f80 0x55de98dd8000 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x55de98edc0c0 tx=0x55de980e8600 comp rx=0 tx=0).stop 2024-09-17T04:34:23.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.751+0000 7ff3550bb740 1 -- 172.21.15.37:0/736272864 shutdown_connections 2024-09-17T04:34:23.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.751+0000 7ff3550bb740 1 --2- 172.21.15.37:0/736272864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55de981a3f80 0x55de98dd8000 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.751+0000 7ff3550bb740 1 --2- 172.21.15.37:0/736272864 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55de981a3b00 0x55de98dd8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.751+0000 7ff3550bb740 1 --2- 172.21.15.37:0/736272864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55de981a3680 0x55de98dd8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.751+0000 7ff3550bb740 1 -- 172.21.15.37:0/736272864 >> 172.21.15.37:0/736272864 conn(0x55de981a4400 msgr2=0x55de9811d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.751+0000 7ff3550bb740 1 -- 172.21.15.37:0/736272864 shutdown_connections 2024-09-17T04:34:23.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.751+0000 7ff3550bb740 1 -- 172.21.15.37:0/736272864 wait complete. 2024-09-17T04:34:23.762 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links', '--filesystem', 'recovery_fs'] in time 0:00:00.108173, result: 2024-09-17T04:34:23.762 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank recovery_fs:0 event recover_dentries list --alternate-pool recovery_fs_metadata 2024-09-17T04:34:23.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.821+0000 7f352340b280 1 -- start start 2024-09-17T04:34:23.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.821+0000 7f352340b280 1 Processor -- start 2024-09-17T04:34:23.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.822+0000 7f352340b280 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559f86cef680 0x559f87a1c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.822+0000 7f352340b280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f86cefb00 0x559f87a1c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.822+0000 7f352340b280 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f86cf0400 0x559f87a1cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.822+0000 7f352340b280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559f87a30600 con 0x559f86cefb00 2024-09-17T04:34:23.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.822+0000 7f352340b280 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559f87a30480 con 0x559f86cf0400 2024-09-17T04:34:23.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.822+0000 7f352340b280 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x559f87a30300 con 0x559f86cef680 2024-09-17T04:34:23.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.822+0000 7f3520646640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f86cefb00 0x559f87a1c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.822+0000 7f3520646640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f86cefb00 0x559f87a1c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38814/0 (socket says 172.21.15.37:38814) 2024-09-17T04:34:23.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.822+0000 7f3520646640 1 -- 172.21.15.37:0/2359845671 learned_addr learned my addr 172.21.15.37:0/2359845671 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.822+0000 7f3520e47640 1 --2- 172.21.15.37:0/2359845671 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559f86cef680 0x559f87a1c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.822+0000 7f3521648640 1 --2- 172.21.15.37:0/2359845671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f86cf0400 0x559f87a1cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.822+0000 7f3520646640 1 -- 172.21.15.37:0/2359845671 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559f86cef680 msgr2=0x559f87a1c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.822+0000 7f3520646640 1 --2- 172.21.15.37:0/2359845671 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559f86cef680 0x559f87a1c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.822+0000 7f3520646640 1 -- 172.21.15.37:0/2359845671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f86cf0400 msgr2=0x559f87a1cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.822+0000 7f3520646640 1 --2- 172.21.15.37:0/2359845671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f86cf0400 0x559f87a1cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.822+0000 7f3520646640 1 -- 172.21.15.37:0/2359845671 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f86c78780 con 0x559f86cefb00 2024-09-17T04:34:23.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.822+0000 7f3520e47640 1 --2- 172.21.15.37:0/2359845671 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559f86cef680 0x559f87a1c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:23.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.822+0000 7f3521648640 1 --2- 172.21.15.37:0/2359845671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f86cf0400 0x559f87a1cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:23.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.823+0000 7f3520646640 1 --2- 172.21.15.37:0/2359845671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f86cefb00 0x559f87a1c580 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x559f87a69350 tx=0x559f86d3a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c93dcdf4cc54a9b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.823+0000 7f351e642640 1 -- 172.21.15.37:0/2359845671 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559f82c30340 con 0x559f86cefb00 2024-09-17T04:34:23.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.823+0000 7f351e642640 1 -- 172.21.15.37:0/2359845671 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x559f82c36fc0 con 0x559f86cefb00 2024-09-17T04:34:23.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.823+0000 7f351e642640 1 -- 172.21.15.37:0/2359845671 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559f82c30ea0 con 0x559f86cefb00 2024-09-17T04:34:23.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.823+0000 7f352340b280 1 -- 172.21.15.37:0/2359845671 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f86c783c0 con 0x559f86cefb00 2024-09-17T04:34:23.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.823+0000 7f352340b280 1 -- 172.21.15.37:0/2359845671 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) -- 0x559f86c79680 con 0x559f86cefb00 2024-09-17T04:34:23.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.823+0000 7f352340b280 4 waiting for MDS map... 2024-09-17T04:34:23.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.823+0000 7f351e642640 1 -- 172.21.15.37:0/2359845671 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x559f87a44000 con 0x559f86cefb00 2024-09-17T04:34:23.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.823+0000 7f351e642640 1 -- 172.21.15.37:0/2359845671 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 4143+0+0 (secure 0 0 0) 0x559f86c95180 con 0x559f86cefb00 2024-09-17T04:34:23.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.823+0000 7f352340b280 4 Got MDS map 22 2024-09-17T04:34:23.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.823+0000 7f352340b280 10 main: JournalTool::main 2024-09-17T04:34:23.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.823+0000 7f352340b280 4 main: JournalTool: connecting to RADOS... 2024-09-17T04:34:23.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.824+0000 7f352340b280 1 Processor -- start 2024-09-17T04:34:23.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.824+0000 7f352340b280 1 -- start start 2024-09-17T04:34:23.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.825+0000 7f352340b280 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559f86ceed80 0x559f87a1d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.825+0000 7f352340b280 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f86cee900 0x559f87a1d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.825+0000 7f352340b280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f86cee480 0x559f87a1db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.825+0000 7f352340b280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559f87a30180 con 0x559f86cee480 2024-09-17T04:34:23.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.825+0000 7f352340b280 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559f86c9d380 con 0x559f86cee900 2024-09-17T04:34:23.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.825+0000 7f352340b280 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x559f86c9d800 con 0x559f86ceed80 2024-09-17T04:34:23.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.825+0000 7f3520646640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f86cee480 0x559f87a1db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.825+0000 7f3520646640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f86cee480 0x559f87a1db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38826/0 (socket says 172.21.15.37:38826) 2024-09-17T04:34:23.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.825+0000 7f3520646640 1 -- 172.21.15.37:0/2121788732 learned_addr learned my addr 172.21.15.37:0/2121788732 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.825+0000 7f3521648640 1 --2- 172.21.15.37:0/2121788732 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559f86ceed80 0x559f87a1d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.826+0000 7f3520e47640 1 --2- 172.21.15.37:0/2121788732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f86cee900 0x559f87a1d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.826+0000 7f3520646640 1 -- 172.21.15.37:0/2121788732 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559f86ceed80 msgr2=0x559f87a1d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.826+0000 7f3520646640 1 --2- 172.21.15.37:0/2121788732 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559f86ceed80 0x559f87a1d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.826+0000 7f3520646640 1 -- 172.21.15.37:0/2121788732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f86cee900 msgr2=0x559f87a1d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.826+0000 7f3520646640 1 --2- 172.21.15.37:0/2121788732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f86cee900 0x559f87a1d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.826+0000 7f3520646640 1 -- 172.21.15.37:0/2121788732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f86c794a0 con 0x559f86cee480 2024-09-17T04:34:23.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.826+0000 7f3520e47640 1 --2- 172.21.15.37:0/2121788732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f86cee900 0x559f87a1d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:23.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.826+0000 7f3520646640 1 --2- 172.21.15.37:0/2121788732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f86cee480 0x559f87a1db80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x559f87a2e360 tx=0x559f87b82120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a69bea97f26b32 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.826+0000 7f351c63e640 1 -- 172.21.15.37:0/2121788732 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559f82c31380 con 0x559f86cee480 2024-09-17T04:34:23.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.826+0000 7f351c63e640 1 -- 172.21.15.37:0/2121788732 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x559f82c36e00 con 0x559f86cee480 2024-09-17T04:34:23.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.826+0000 7f351c63e640 1 -- 172.21.15.37:0/2121788732 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559f82c311e0 con 0x559f86cee480 2024-09-17T04:34:23.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.827+0000 7f352340b280 1 -- 172.21.15.37:0/2121788732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f86cee480 msgr2=0x559f87a1db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.827+0000 7f352340b280 1 --2- 172.21.15.37:0/2121788732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f86cee480 0x559f87a1db80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x559f87a2e360 tx=0x559f87b82120 comp rx=0 tx=0).stop 2024-09-17T04:34:23.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.827+0000 7f352340b280 1 -- 172.21.15.37:0/2121788732 shutdown_connections 2024-09-17T04:34:23.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.827+0000 7f352340b280 1 --2- 172.21.15.37:0/2121788732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f86cee480 0x559f87a1db80 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.827+0000 7f352340b280 1 --2- 172.21.15.37:0/2121788732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f86cee900 0x559f87a1d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.827+0000 7f352340b280 1 --2- 172.21.15.37:0/2121788732 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559f86ceed80 0x559f87a1d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.827+0000 7f352340b280 1 -- 172.21.15.37:0/2121788732 >> 172.21.15.37:0/2121788732 conn(0x559f86cef200 msgr2=0x559f86d63800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.827+0000 7f352340b280 1 -- 172.21.15.37:0/2121788732 shutdown_connections 2024-09-17T04:34:23.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.827+0000 7f352340b280 1 -- 172.21.15.37:0/2121788732 wait complete. 2024-09-17T04:34:23.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.827+0000 7f352340b280 1 Processor -- start 2024-09-17T04:34:23.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.827+0000 7f352340b280 1 -- start start 2024-09-17T04:34:23.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.829+0000 7f352340b280 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559f86cee480 0x559f87a1db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.829+0000 7f352340b280 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f86cee900 0x559f87a1d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.829+0000 7f352340b280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f86ceed80 0x559f87a1d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.829+0000 7f352340b280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559f87a30c00 con 0x559f86ceed80 2024-09-17T04:34:23.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.829+0000 7f352340b280 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559f87a30a80 con 0x559f86cee900 2024-09-17T04:34:23.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.829+0000 7f352340b280 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x559f87a31380 con 0x559f86cee480 2024-09-17T04:34:23.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.829+0000 7f3520646640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f86ceed80 0x559f87a1d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.829+0000 7f3520646640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f86ceed80 0x559f87a1d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38838/0 (socket says 172.21.15.37:38838) 2024-09-17T04:34:23.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.829+0000 7f3521648640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559f86cee480 0x559f87a1db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.829+0000 7f3520646640 1 -- 172.21.15.37:0/1968378762 learned_addr learned my addr 172.21.15.37:0/1968378762 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.829+0000 7f3520e47640 1 --2- 172.21.15.37:0/1968378762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f86cee900 0x559f87a1d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.829+0000 7f3520646640 1 -- 172.21.15.37:0/1968378762 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559f86cee480 msgr2=0x559f87a1db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.829+0000 7f3520646640 1 --2- 172.21.15.37:0/1968378762 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559f86cee480 0x559f87a1db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.829+0000 7f3520646640 1 -- 172.21.15.37:0/1968378762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f86cee900 msgr2=0x559f87a1d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.829+0000 7f3520646640 1 --2- 172.21.15.37:0/1968378762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f86cee900 0x559f87a1d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.829+0000 7f3520646640 1 -- 172.21.15.37:0/1968378762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f86c78000 con 0x559f86ceed80 2024-09-17T04:34:23.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.829+0000 7f3520646640 1 --2- 172.21.15.37:0/1968378762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f86ceed80 0x559f87a1d080 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x559f87a2f4d0 tx=0x559f87b82300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.829+0000 7f351ae3b640 1 -- 172.21.15.37:0/1968378762 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559f82c31380 con 0x559f86ceed80 2024-09-17T04:34:23.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.829+0000 7f351ae3b640 1 -- 172.21.15.37:0/1968378762 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x559f82c36700 con 0x559f86ceed80 2024-09-17T04:34:23.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.829+0000 7f352340b280 1 -- 172.21.15.37:0/1968378762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559f86c78960 con 0x559f86ceed80 2024-09-17T04:34:23.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.829+0000 7f351ae3b640 1 -- 172.21.15.37:0/1968378762 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559f82c311e0 con 0x559f86ceed80 2024-09-17T04:34:23.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.829+0000 7f352340b280 1 -- 172.21.15.37:0/1968378762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f86c79860 con 0x559f86ceed80 2024-09-17T04:34:23.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.829+0000 7f352340b280 4 main: JournalTool: resolving pool 6 2024-09-17T04:34:23.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.831+0000 7f351ae3b640 1 -- 172.21.15.37:0/1968378762 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x559f87b78900 con 0x559f86ceed80 2024-09-17T04:34:23.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.831+0000 7f351ae3b640 1 --2- 172.21.15.37:0/1968378762 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x559f86cf1180 0x559f87a1e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.831+0000 7f351ae3b640 1 -- 172.21.15.37:0/1968378762 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x559f87b86c00 con 0x559f86ceed80 2024-09-17T04:34:23.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.831+0000 7f352340b280 4 main: JournalTool: creating IoCtx.. 2024-09-17T04:34:23.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.831+0000 7f352340b280 4 main: Executing for rank 0 2024-09-17T04:34:23.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.831+0000 7f352340b280 1 main_event: Using alternate pool recovery_fs_metadata 2024-09-17T04:34:23.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.831+0000 7f3521648640 1 --2- 172.21.15.37:0/1968378762 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x559f86cf1180 0x559f87a1e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.831+0000 7f352340b280 1 --2- 172.21.15.37:0/1968378762 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x559f86cee000 0x559f87a1e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.831+0000 7f352340b280 1 -- 172.21.15.37:0/1968378762 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:1 6.f 6:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x559f87b89800 con 0x559f86cee000 2024-09-17T04:34:23.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.832+0000 7f3520e47640 1 --2- 172.21.15.37:0/1968378762 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x559f86cee000 0x559f87a1e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.832+0000 7f3521648640 1 --2- 172.21.15.37:0/1968378762 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x559f86cf1180 0x559f87a1e100 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x559f87a69c80 tx=0x559f87b82780 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.832+0000 7f3520e47640 1 --2- 172.21.15.37:0/1968378762 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x559f86cee000 0x559f87a1e680 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.832+0000 7f3520e47640 1 -- 172.21.15.37:0/1968378762 <== osd.2 v2:172.21.15.37:6816/579720176 1 ==== osd_op_reply(1 400.00000000 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x559f86d7e000 con 0x559f86cee000 2024-09-17T04:34:23.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.832+0000 7f352340b280 -1 Pointer 400.00000000 is absent 2024-09-17T04:34:23.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.832+0000 7f352340b280 4 main_event: consumed 0 inodes 2024-09-17T04:34:23.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.832+0000 7f352340b280 1 --2- 172.21.15.37:0/1968378762 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x559f86cf0880 0x559f87a1ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.832+0000 7f352340b280 1 -- 172.21.15.37:0/1968378762 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- osd_op(unknown.0.0:2 6.3 6:c0ce80d4:::4.00000000:head [omap-rm-keys in=4b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x559f87b89c00 con 0x559f86cf0880 2024-09-17T04:34:23.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.832+0000 7f3521648640 1 --2- 172.21.15.37:0/1968378762 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x559f86cf0880 0x559f87a1ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.833+0000 7f3521648640 1 --2- 172.21.15.37:0/1968378762 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x559f86cf0880 0x559f87a1ec00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.834+0000 7f3521648640 1 -- 172.21.15.37:0/1968378762 <== osd.7 v2:172.21.15.92:6802/3444787223 1 ==== osd_op_reply(2 4.00000000 [omap-rm-keys] v41'1 uv0 ondisk = -2 ((2) No such file or directory)) ==== 154+0+0 (crc 0 0 0) 0x559f86d7e500 con 0x559f86cf0880 2024-09-17T04:34:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.834+0000 7f352340b280 4 0 events 2024-09-17T04:34:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.834+0000 7f352340b280 1 -- 172.21.15.37:0/1968378762 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x559f86cee000 msgr2=0x559f87a1e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.834+0000 7f352340b280 1 --2- 172.21.15.37:0/1968378762 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x559f86cee000 0x559f87a1e680 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.834+0000 7f352340b280 1 -- 172.21.15.37:0/1968378762 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x559f86cf0880 msgr2=0x559f87a1ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.834+0000 7f352340b280 1 --2- 172.21.15.37:0/1968378762 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x559f86cf0880 0x559f87a1ec00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.834+0000 7f352340b280 1 -- 172.21.15.37:0/1968378762 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x559f86cf1180 msgr2=0x559f87a1e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.834+0000 7f352340b280 1 --2- 172.21.15.37:0/1968378762 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x559f86cf1180 0x559f87a1e100 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x559f87a69c80 tx=0x559f87b82780 comp rx=0 tx=0).stop 2024-09-17T04:34:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.834+0000 7f352340b280 1 -- 172.21.15.37:0/1968378762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f86ceed80 msgr2=0x559f87a1d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.834+0000 7f352340b280 1 --2- 172.21.15.37:0/1968378762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f86ceed80 0x559f87a1d080 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x559f87a2f4d0 tx=0x559f87b82300 comp rx=0 tx=0).stop 2024-09-17T04:34:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.834+0000 7f3520e47640 1 -- 172.21.15.37:0/1968378762 reap_dead start 2024-09-17T04:34:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.834+0000 7f352340b280 1 -- 172.21.15.37:0/1968378762 shutdown_connections 2024-09-17T04:34:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.834+0000 7f352340b280 1 -- 172.21.15.37:0/1968378762 >> 172.21.15.37:0/1968378762 conn(0x559f86cef200 msgr2=0x559f86d63800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.834+0000 7f352340b280 1 -- 172.21.15.37:0/1968378762 shutdown_connections 2024-09-17T04:34:23.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.834+0000 7f352340b280 1 -- 172.21.15.37:0/1968378762 wait complete. 2024-09-17T04:34:23.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.835+0000 7f352340b280 1 -- 172.21.15.37:0/2359845671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f86cefb00 msgr2=0x559f87a1c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.835+0000 7f352340b280 1 --2- 172.21.15.37:0/2359845671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f86cefb00 0x559f87a1c580 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x559f87a69350 tx=0x559f86d3a600 comp rx=0 tx=0).stop 2024-09-17T04:34:23.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.835+0000 7f352340b280 1 -- 172.21.15.37:0/2359845671 shutdown_connections 2024-09-17T04:34:23.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.835+0000 7f352340b280 1 --2- 172.21.15.37:0/2359845671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559f86cf0400 0x559f87a1cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.835+0000 7f352340b280 1 --2- 172.21.15.37:0/2359845671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f86cefb00 0x559f87a1c580 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.835+0000 7f352340b280 1 --2- 172.21.15.37:0/2359845671 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559f86cef680 0x559f87a1c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.835+0000 7f352340b280 1 -- 172.21.15.37:0/2359845671 >> 172.21.15.37:0/2359845671 conn(0x559f86ceff80 msgr2=0x559f86d61800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.835+0000 7f352340b280 1 -- 172.21.15.37:0/2359845671 shutdown_connections 2024-09-17T04:34:23.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.835+0000 7f352340b280 1 -- 172.21.15.37:0/2359845671 wait complete. 2024-09-17T04:34:23.841 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'recovery_fs:0', 'event', 'recover_dentries', 'list', '--alternate-pool', 'recovery_fs_metadata'] in time 0:00:00.078976, result: 2024-09-17T04:34:23.841 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank recovery_fs:0 journal reset --force --yes-i-really-really-mean-it 2024-09-17T04:34:23.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.897+0000 7ff76ad21280 1 -- start start 2024-09-17T04:34:23.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.897+0000 7ff76ad21280 1 Processor -- start 2024-09-17T04:34:23.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.899+0000 7ff76ad21280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb59680 0x5585ed87a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.899+0000 7ff76ad21280 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5585ecb59b00 0x5585ed87a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.899+0000 7ff76ad21280 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585ecb5a400 0x5585ed87ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.899+0000 7ff76ad21280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5585ed976180 con 0x5585ecb59680 2024-09-17T04:34:23.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.899+0000 7ff76ad21280 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5585ed976300 con 0x5585ecb5a400 2024-09-17T04:34:23.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.899+0000 7ff76ad21280 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x5585ed976480 con 0x5585ecb59b00 2024-09-17T04:34:23.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.899+0000 7ff768a5f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb59680 0x5585ed87a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.899+0000 7ff768a5f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb59680 0x5585ed87a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38852/0 (socket says 172.21.15.37:38852) 2024-09-17T04:34:23.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.899+0000 7ff768a5f640 1 -- 172.21.15.37:0/3726026532 learned_addr learned my addr 172.21.15.37:0/3726026532 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.899+0000 7ff76825e640 1 --2- 172.21.15.37:0/3726026532 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5585ecb59b00 0x5585ed87a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.899+0000 7ff769260640 1 --2- 172.21.15.37:0/3726026532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585ecb5a400 0x5585ed87ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.899+0000 7ff768a5f640 1 -- 172.21.15.37:0/3726026532 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5585ecb59b00 msgr2=0x5585ed87a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.899+0000 7ff768a5f640 1 --2- 172.21.15.37:0/3726026532 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5585ecb59b00 0x5585ed87a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.899+0000 7ff768a5f640 1 -- 172.21.15.37:0/3726026532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585ecb5a400 msgr2=0x5585ed87ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.899+0000 7ff768a5f640 1 --2- 172.21.15.37:0/3726026532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585ecb5a400 0x5585ed87ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.899+0000 7ff768a5f640 1 -- 172.21.15.37:0/3726026532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585ecae2780 con 0x5585ecb59680 2024-09-17T04:34:23.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.899+0000 7ff769260640 1 --2- 172.21.15.37:0/3726026532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585ecb5a400 0x5585ed87ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:23.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.899+0000 7ff768a5f640 1 --2- 172.21.15.37:0/3726026532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb59680 0x5585ed87a580 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x5585ed8d3bf0 tx=0x5585ecba6b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e835031c3fedfa6a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.899+0000 7ff76625a640 1 -- 172.21.15.37:0/3726026532 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5585e8a9aea0 con 0x5585ecb59680 2024-09-17T04:34:23.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.900+0000 7ff76625a640 1 -- 172.21.15.37:0/3726026532 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5585e8aa1180 con 0x5585ecb59680 2024-09-17T04:34:23.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.900+0000 7ff76625a640 1 -- 172.21.15.37:0/3726026532 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5585e8a9b380 con 0x5585ecb59680 2024-09-17T04:34:23.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.900+0000 7ff76ad21280 1 -- 172.21.15.37:0/3726026532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585ecae23c0 con 0x5585ecb59680 2024-09-17T04:34:23.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.900+0000 7ff76ad21280 1 -- 172.21.15.37:0/3726026532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5585ecae3680 con 0x5585ecb59680 2024-09-17T04:34:23.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.900+0000 7ff76ad21280 4 waiting for MDS map... 2024-09-17T04:34:23.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.900+0000 7ff76625a640 1 -- 172.21.15.37:0/3726026532 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x5585ed98a600 con 0x5585ecb59680 2024-09-17T04:34:23.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.900+0000 7ff76625a640 1 -- 172.21.15.37:0/3726026532 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 4143+0+0 (secure 0 0 0) 0x5585ecaff180 con 0x5585ecb59680 2024-09-17T04:34:23.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.900+0000 7ff76ad21280 4 Got MDS map 22 2024-09-17T04:34:23.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.900+0000 7ff76ad21280 10 main: JournalTool::main 2024-09-17T04:34:23.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.900+0000 7ff76ad21280 4 main: JournalTool: connecting to RADOS... 2024-09-17T04:34:23.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.901+0000 7ff76ad21280 1 Processor -- start 2024-09-17T04:34:23.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.901+0000 7ff76ad21280 1 -- start start 2024-09-17T04:34:23.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.902+0000 7ff76ad21280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb58d80 0x5585ed87b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.902+0000 7ff76ad21280 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5585ecb58900 0x5585ed87b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.902+0000 7ff76ad21280 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585ecb58480 0x5585ed87bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.902+0000 7ff76ad21280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5585ed976600 con 0x5585ecb58d80 2024-09-17T04:34:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.902+0000 7ff76ad21280 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5585ed976780 con 0x5585ecb58480 2024-09-17T04:34:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.902+0000 7ff76ad21280 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x5585ed976900 con 0x5585ecb58900 2024-09-17T04:34:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.902+0000 7ff769260640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb58d80 0x5585ed87b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.902+0000 7ff769260640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb58d80 0x5585ed87b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38866/0 (socket says 172.21.15.37:38866) 2024-09-17T04:34:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.902+0000 7ff769260640 1 -- 172.21.15.37:0/4144353595 learned_addr learned my addr 172.21.15.37:0/4144353595 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.902+0000 7ff769260640 1 -- 172.21.15.37:0/4144353595 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5585ecb58900 msgr2=0x5585ed87b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.902+0000 7ff768a5f640 1 --2- 172.21.15.37:0/4144353595 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5585ecb58900 0x5585ed87b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.903+0000 7ff769260640 1 --2- 172.21.15.37:0/4144353595 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5585ecb58900 0x5585ed87b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.903+0000 7ff76825e640 1 --2- 172.21.15.37:0/4144353595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585ecb58480 0x5585ed87bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.903+0000 7ff769260640 1 -- 172.21.15.37:0/4144353595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585ecb58480 msgr2=0x5585ed87bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.903+0000 7ff769260640 1 --2- 172.21.15.37:0/4144353595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585ecb58480 0x5585ed87bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.903+0000 7ff769260640 1 -- 172.21.15.37:0/4144353595 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585ecae34a0 con 0x5585ecb58d80 2024-09-17T04:34:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.903+0000 7ff769260640 1 --2- 172.21.15.37:0/4144353595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb58d80 0x5585ed87b080 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x5585ed9a4fc0 tx=0x5585ed9aec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8110610a30667ad2 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.903+0000 7ff764256640 1 -- 172.21.15.37:0/4144353595 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5585e8a9aea0 con 0x5585ecb58d80 2024-09-17T04:34:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.903+0000 7ff764256640 1 -- 172.21.15.37:0/4144353595 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5585e8aa0380 con 0x5585ecb58d80 2024-09-17T04:34:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.903+0000 7ff764256640 1 -- 172.21.15.37:0/4144353595 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5585e8a9b380 con 0x5585ecb58d80 2024-09-17T04:34:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.903+0000 7ff76ad21280 1 -- 172.21.15.37:0/4144353595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb58d80 msgr2=0x5585ed87b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.903+0000 7ff76ad21280 1 --2- 172.21.15.37:0/4144353595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb58d80 0x5585ed87b080 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x5585ed9a4fc0 tx=0x5585ed9aec00 comp rx=0 tx=0).stop 2024-09-17T04:34:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.904+0000 7ff76ad21280 1 -- 172.21.15.37:0/4144353595 shutdown_connections 2024-09-17T04:34:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.904+0000 7ff76ad21280 1 --2- 172.21.15.37:0/4144353595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585ecb58480 0x5585ed87bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.904+0000 7ff76ad21280 1 --2- 172.21.15.37:0/4144353595 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5585ecb58900 0x5585ed87b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.904+0000 7ff76ad21280 1 --2- 172.21.15.37:0/4144353595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb58d80 0x5585ed87b080 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.904+0000 7ff76ad21280 1 -- 172.21.15.37:0/4144353595 >> 172.21.15.37:0/4144353595 conn(0x5585ecb59200 msgr2=0x5585ecbcd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.904+0000 7ff76ad21280 1 -- 172.21.15.37:0/4144353595 shutdown_connections 2024-09-17T04:34:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.904+0000 7ff76ad21280 1 -- 172.21.15.37:0/4144353595 wait complete. 2024-09-17T04:34:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.904+0000 7ff76ad21280 1 Processor -- start 2024-09-17T04:34:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.904+0000 7ff76ad21280 1 -- start start 2024-09-17T04:34:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.906+0000 7ff76ad21280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb58480 0x5585ed87bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.906+0000 7ff76ad21280 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585ecb58900 0x5585ed87b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.906+0000 7ff76ad21280 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5585ecb58d80 0x5585ed87b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.906+0000 7ff76ad21280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5585ed976900 con 0x5585ecb58480 2024-09-17T04:34:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.906+0000 7ff76ad21280 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5585ed976a80 con 0x5585ecb58900 2024-09-17T04:34:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.906+0000 7ff76ad21280 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x5585ed976780 con 0x5585ecb58d80 2024-09-17T04:34:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.906+0000 7ff769260640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb58480 0x5585ed87bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.906+0000 7ff769260640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb58480 0x5585ed87bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38874/0 (socket says 172.21.15.37:38874) 2024-09-17T04:34:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.906+0000 7ff769260640 1 -- 172.21.15.37:0/4161761778 learned_addr learned my addr 172.21.15.37:0/4161761778 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.906+0000 7ff768a5f640 1 --2- 172.21.15.37:0/4161761778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585ecb58900 0x5585ed87b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.906+0000 7ff76825e640 1 --2- 172.21.15.37:0/4161761778 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5585ecb58d80 0x5585ed87b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.906+0000 7ff769260640 1 -- 172.21.15.37:0/4161761778 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5585ecb58d80 msgr2=0x5585ed87b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.906+0000 7ff769260640 1 --2- 172.21.15.37:0/4161761778 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5585ecb58d80 0x5585ed87b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.906+0000 7ff769260640 1 -- 172.21.15.37:0/4161761778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585ecb58900 msgr2=0x5585ed87b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.906+0000 7ff769260640 1 --2- 172.21.15.37:0/4161761778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585ecb58900 0x5585ed87b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.906+0000 7ff769260640 1 -- 172.21.15.37:0/4161761778 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585ecae2780 con 0x5585ecb58480 2024-09-17T04:34:23.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.906+0000 7ff769260640 1 --2- 172.21.15.37:0/4161761778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb58480 0x5585ed87bb80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x5585ed9a5c80 tx=0x5585ed9aede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.906+0000 7ff762a53640 1 -- 172.21.15.37:0/4161761778 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5585e8a9b040 con 0x5585ecb58480 2024-09-17T04:34:23.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.906+0000 7ff762a53640 1 -- 172.21.15.37:0/4161761778 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5585e8aa0fc0 con 0x5585ecb58480 2024-09-17T04:34:23.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.906+0000 7ff762a53640 1 -- 172.21.15.37:0/4161761778 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5585e8a9aea0 con 0x5585ecb58480 2024-09-17T04:34:23.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.906+0000 7ff76ad21280 1 -- 172.21.15.37:0/4161761778 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5585ecae23c0 con 0x5585ecb58480 2024-09-17T04:34:23.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.906+0000 7ff76ad21280 1 -- 172.21.15.37:0/4161761778 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585ecae2000 con 0x5585ecb58480 2024-09-17T04:34:23.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.906+0000 7ff76ad21280 4 main: JournalTool: resolving pool 6 2024-09-17T04:34:23.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.908+0000 7ff762a53640 1 -- 172.21.15.37:0/4161761778 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x5585ed9e2900 con 0x5585ecb58480 2024-09-17T04:34:23.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.908+0000 7ff762a53640 1 --2- 172.21.15.37:0/4161761778 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x5585ecb5b180 0x5585ed87c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.908+0000 7ff762a53640 1 -- 172.21.15.37:0/4161761778 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x5585eda00a00 con 0x5585ecb58480 2024-09-17T04:34:23.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.908+0000 7ff76ad21280 4 main: JournalTool: creating IoCtx.. 2024-09-17T04:34:23.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.908+0000 7ff76ad21280 4 main: Executing for rank 0 2024-09-17T04:34:23.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.908+0000 7ff768a5f640 1 --2- 172.21.15.37:0/4161761778 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x5585ecb5b180 0x5585ed87c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.909+0000 7ff768a5f640 1 --2- 172.21.15.37:0/4161761778 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x5585ecb5b180 0x5585ed87c100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x5585ed9a4180 tx=0x5585ecb80540 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.909+0000 7ff76ad21280 1 -- start start 2024-09-17T04:34:23.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.909+0000 7ff76ad21280 1 Processor -- start 2024-09-17T04:34:23.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.910+0000 7ff76ad21280 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585ecb5a880 0x5585ed87c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.910+0000 7ff76ad21280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb5ad00 0x5585ed87cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.910+0000 7ff76ad21280 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5585ecb5b600 0x5585ed87d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.910+0000 7ff76ad21280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5585ed977b00 con 0x5585ecb5ad00 2024-09-17T04:34:23.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.910+0000 7ff76ad21280 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5585ed977080 con 0x5585ecb5a880 2024-09-17T04:34:23.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.910+0000 7ff76ad21280 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x5585ed976f00 con 0x5585ecb5b600 2024-09-17T04:34:23.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.911+0000 7ff768a5f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb5ad00 0x5585ed87cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.911+0000 7ff768a5f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb5ad00 0x5585ed87cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38878/0 (socket says 172.21.15.37:38878) 2024-09-17T04:34:23.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.911+0000 7ff768a5f640 1 -- 172.21.15.37:0/2404798951 learned_addr learned my addr 172.21.15.37:0/2404798951 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:23.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.911+0000 7ff76825e640 1 --2- 172.21.15.37:0/2404798951 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5585ecb5b600 0x5585ed87d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.911+0000 7ff769260640 1 --2- 172.21.15.37:0/2404798951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585ecb5a880 0x5585ed87c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.911+0000 7ff768a5f640 1 -- 172.21.15.37:0/2404798951 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5585ecb5b600 msgr2=0x5585ed87d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.911+0000 7ff768a5f640 1 --2- 172.21.15.37:0/2404798951 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5585ecb5b600 0x5585ed87d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.911+0000 7ff768a5f640 1 -- 172.21.15.37:0/2404798951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585ecb5a880 msgr2=0x5585ed87c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:23.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.911+0000 7ff768a5f640 1 --2- 172.21.15.37:0/2404798951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585ecb5a880 0x5585ed87c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.911+0000 7ff768a5f640 1 -- 172.21.15.37:0/2404798951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585ecae2960 con 0x5585ecb5ad00 2024-09-17T04:34:23.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.911+0000 7ff76825e640 1 --2- 172.21.15.37:0/2404798951 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5585ecb5b600 0x5585ed87d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T04:34:23.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.911+0000 7ff768a5f640 1 --2- 172.21.15.37:0/2404798951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb5ad00 0x5585ed87cc00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x5585ecf04090 tx=0x5585ecb805a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8fc40ba4e0d9479 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.911+0000 7ff75fa4d640 1 -- 172.21.15.37:0/2404798951 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5585e8a9b1e0 con 0x5585ecb5ad00 2024-09-17T04:34:23.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.911+0000 7ff75fa4d640 1 -- 172.21.15.37:0/2404798951 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5585e8aa0e00 con 0x5585ecb5ad00 2024-09-17T04:34:23.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.911+0000 7ff76ad21280 1 -- 172.21.15.37:0/2404798951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585ecae34a0 con 0x5585ecb5ad00 2024-09-17T04:34:23.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.911+0000 7ff75fa4d640 1 -- 172.21.15.37:0/2404798951 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5585eda224e0 con 0x5585ecb5ad00 2024-09-17T04:34:23.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.911+0000 7ff76ad21280 1 -- 172.21.15.37:0/2404798951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5585ecae2780 con 0x5585ecb5ad00 2024-09-17T04:34:23.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.911+0000 7ff76ad21280 4 waiting for MDS map... 2024-09-17T04:34:23.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.911+0000 7ff75fa4d640 1 -- 172.21.15.37:0/2404798951 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x5585ecf06600 con 0x5585ecb5ad00 2024-09-17T04:34:23.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.912+0000 7ff75fa4d640 1 -- 172.21.15.37:0/2404798951 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 4143+0+0 (secure 0 0 0) 0x5585ecaff500 con 0x5585ecb5ad00 2024-09-17T04:34:23.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.912+0000 7ff76ad21280 4 Got MDS map 22 2024-09-17T04:34:23.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.912+0000 7ff76ad21280 1 --2- 172.21.15.37:0/2404798951 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x5585ecb5ba80 0x5585ed87d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.912+0000 7ff76ad21280 1 -- 172.21.15.37:0/2404798951 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:1 6.f 6:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x5585ecf20000 con 0x5585ecb5ba80 2024-09-17T04:34:23.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.912+0000 7ff769260640 1 --2- 172.21.15.37:0/2404798951 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x5585ecb5ba80 0x5585ed87d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.912+0000 7ff769260640 1 --2- 172.21.15.37:0/2404798951 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x5585ecb5ba80 0x5585ed87d700 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=499796fbd815a948 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.914+0000 7ff769260640 1 -- 172.21.15.37:0/2404798951 <== osd.2 v2:172.21.15.37:6816/579720176 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v41'1 uv1 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x5585ecbe8000 con 0x5585ecb5ba80 2024-09-17T04:34:23.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.914+0000 7ff76ad21280 1 -- 172.21.15.37:0/2404798951 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:2 6.14 6:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x5585ecf20400 con 0x5585ecb5ba80 2024-09-17T04:34:23.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.915+0000 7ff769260640 1 -- 172.21.15.37:0/2404798951 <== osd.2 v2:172.21.15.37:6816/579720176 2 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v41'2 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x5585ecbe8000 con 0x5585ecb5ba80 2024-09-17T04:34:23.917 INFO:teuthology.orchestra.run.smithi037.stdout:writing EResetJournal entry 2024-09-17T04:34:23.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.916+0000 7ff76ad21280 1 -- 172.21.15.37:0/2404798951 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:3 6.14 6:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x5585ecf20800 con 0x5585ecb5ba80 2024-09-17T04:34:23.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.916+0000 7ff76ad21280 1 --2- 172.21.15.37:0/2404798951 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x5585ecf2a000 0x5585ecf32000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.916+0000 7ff76ad21280 1 -- 172.21.15.37:0/2404798951 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:4 6.1b 6:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x5585ecf20c00 con 0x5585ecf2a000 2024-09-17T04:34:23.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.916+0000 7ff76ad21280 1 -- 172.21.15.37:0/2404798951 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:5 6.b 6:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x5585ecf21000 con 0x5585ecf2a000 2024-09-17T04:34:23.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.916+0000 7ff76ad21280 1 -- 172.21.15.37:0/2404798951 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- osd_op(unknown.0.0:6 6.1f 6:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x5585ecf21400 con 0x5585ecf2a000 2024-09-17T04:34:23.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.916+0000 7ff76ad21280 1 --2- 172.21.15.37:0/2404798951 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x5585ecf2a480 0x5585ecf32580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:23.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.916+0000 7ff76ad21280 1 -- 172.21.15.37:0/2404798951 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:7 6.1a 6:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x5585ecf21800 con 0x5585ecf2a480 2024-09-17T04:34:23.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.916+0000 7ff76ad21280 1 -- 172.21.15.37:0/2404798951 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- osd_op(unknown.0.0:8 6.1e 6:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x5585ecf21c00 con 0x5585ecf2a480 2024-09-17T04:34:23.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.916+0000 7ff76825e640 1 --2- 172.21.15.37:0/2404798951 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x5585ecf2a000 0x5585ecf32000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.916+0000 7ff769260640 1 --2- 172.21.15.37:0/2404798951 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x5585ecf2a480 0x5585ecf32580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:23.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.916+0000 7ff76825e640 1 --2- 172.21.15.37:0/2404798951 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x5585ecf2a000 0x5585ecf32000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=e0e4906cf7fcfdae server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.917+0000 7ff769260640 1 --2- 172.21.15.37:0/2404798951 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x5585ecf2a480 0x5585ecf32580 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=d511cffd349842d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:23.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.917+0000 7ff769260640 1 -- 172.21.15.37:0/2404798951 <== osd.2 v2:172.21.15.37:6816/579720176 3 ==== osd_op_reply(3 200.00000001 [delete] v41'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5585ecbe8000 con 0x5585ecb5ba80 2024-09-17T04:34:23.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.918+0000 7ff76825e640 1 -- 172.21.15.37:0/2404798951 <== osd.3 v2:172.21.15.37:6800/809975826 1 ==== osd_op_reply(4 200.00000002 [delete] v41'1 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5585ecbe8500 con 0x5585ecf2a000 2024-09-17T04:34:23.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.918+0000 7ff75e24a640 1 -- 172.21.15.37:0/2404798951 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- osd_op(unknown.0.0:9 6.14 6:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41) -- 0x5585ecf20000 con 0x5585ecb5ba80 2024-09-17T04:34:23.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.918+0000 7ff76825e640 1 -- 172.21.15.37:0/2404798951 <== osd.3 v2:172.21.15.37:6800/809975826 2 ==== osd_op_reply(6 200.00000004 [delete] v41'5 uv4 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5585ecbe8500 con 0x5585ecf2a000 2024-09-17T04:34:23.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.918+0000 7ff769260640 1 -- 172.21.15.37:0/2404798951 <== osd.4 v2:172.21.15.92:6818/2509785752 1 ==== osd_op_reply(8 200.00000006 [delete] v41'1 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5585ecbe8000 con 0x5585ecf2a480 2024-09-17T04:34:23.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.918+0000 7ff76825e640 1 -- 172.21.15.37:0/2404798951 <== osd.3 v2:172.21.15.37:6800/809975826 3 ==== osd_op_reply(5 200.00000003 [delete] v41'2 uv1 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5585ecbe8500 con 0x5585ecf2a000 2024-09-17T04:34:23.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.918+0000 7ff769260640 1 -- 172.21.15.37:0/2404798951 <== osd.4 v2:172.21.15.92:6818/2509785752 2 ==== osd_op_reply(7 200.00000005 [delete] v41'1 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5585ecbe8000 con 0x5585ecf2a480 2024-09-17T04:34:23.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.919+0000 7ff769260640 1 -- 172.21.15.37:0/2404798951 <== osd.2 v2:172.21.15.37:6816/579720176 4 ==== osd_op_reply(9 200.00000001 [write 0~48] v41'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x5585ecbe8000 con 0x5585ecb5ba80 2024-09-17T04:34:23.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.919+0000 7ff76ad21280 4 Successfully wrote new journal pointer and header for rank 3:0 2024-09-17T04:34:23.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.919+0000 7ff76ad21280 1 -- 172.21.15.37:0/2404798951 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x5585ecb5ba80 msgr2=0x5585ed87d700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.919+0000 7ff76ad21280 1 --2- 172.21.15.37:0/2404798951 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x5585ecb5ba80 0x5585ed87d700 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.919+0000 7ff76ad21280 1 -- 172.21.15.37:0/2404798951 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x5585ecf2a000 msgr2=0x5585ecf32000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.919+0000 7ff76ad21280 1 --2- 172.21.15.37:0/2404798951 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x5585ecf2a000 0x5585ecf32000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.919+0000 7ff76ad21280 1 -- 172.21.15.37:0/2404798951 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x5585ecf2a480 msgr2=0x5585ecf32580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.919+0000 7ff76ad21280 1 --2- 172.21.15.37:0/2404798951 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x5585ecf2a480 0x5585ecf32580 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.920+0000 7ff76825e640 1 -- 172.21.15.37:0/2404798951 reap_dead start 2024-09-17T04:34:23.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.920+0000 7ff76ad21280 1 -- 172.21.15.37:0/2404798951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb5ad00 msgr2=0x5585ed87cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.920+0000 7ff76ad21280 1 --2- 172.21.15.37:0/2404798951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb5ad00 0x5585ed87cc00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x5585ecf04090 tx=0x5585ecb805a0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.920+0000 7ff76ad21280 1 -- 172.21.15.37:0/2404798951 shutdown_connections 2024-09-17T04:34:23.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.920+0000 7ff76ad21280 1 --2- 172.21.15.37:0/2404798951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb5ad00 0x5585ed87cc00 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.920+0000 7ff76ad21280 1 -- 172.21.15.37:0/2404798951 >> 172.21.15.37:0/2404798951 conn(0x5585ecb58000 msgr2=0x5585eceaa800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.920+0000 7ff76ad21280 1 -- 172.21.15.37:0/2404798951 shutdown_connections 2024-09-17T04:34:23.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.920+0000 7ff76ad21280 1 -- 172.21.15.37:0/2404798951 wait complete. 2024-09-17T04:34:23.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.920+0000 7ff76ad21280 1 -- 172.21.15.37:0/4161761778 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x5585ecb5b180 msgr2=0x5585ed87c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.920+0000 7ff76ad21280 1 --2- 172.21.15.37:0/4161761778 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x5585ecb5b180 0x5585ed87c100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x5585ed9a4180 tx=0x5585ecb80540 comp rx=0 tx=0).stop 2024-09-17T04:34:23.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.920+0000 7ff76ad21280 1 -- 172.21.15.37:0/4161761778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb58480 msgr2=0x5585ed87bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.920+0000 7ff76ad21280 1 --2- 172.21.15.37:0/4161761778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb58480 0x5585ed87bb80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x5585ed9a5c80 tx=0x5585ed9aede0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.920+0000 7ff76ad21280 1 -- 172.21.15.37:0/4161761778 shutdown_connections 2024-09-17T04:34:23.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.920+0000 7ff76ad21280 1 --2- 172.21.15.37:0/4161761778 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x5585ecb5b180 0x5585ed87c100 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.920+0000 7ff76ad21280 1 --2- 172.21.15.37:0/4161761778 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5585ecb58d80 0x5585ed87b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.920+0000 7ff76ad21280 1 --2- 172.21.15.37:0/4161761778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585ecb58900 0x5585ed87b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.920+0000 7ff76ad21280 1 --2- 172.21.15.37:0/4161761778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb58480 0x5585ed87bb80 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.920+0000 7ff76ad21280 1 -- 172.21.15.37:0/4161761778 >> 172.21.15.37:0/4161761778 conn(0x5585ecb59200 msgr2=0x5585ecbcd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.920+0000 7ff76ad21280 1 -- 172.21.15.37:0/4161761778 shutdown_connections 2024-09-17T04:34:23.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.920+0000 7ff76ad21280 1 -- 172.21.15.37:0/4161761778 wait complete. 2024-09-17T04:34:23.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.920+0000 7ff76ad21280 1 -- 172.21.15.37:0/3726026532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb59680 msgr2=0x5585ed87a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:23.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.920+0000 7ff76ad21280 1 --2- 172.21.15.37:0/3726026532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb59680 0x5585ed87a580 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x5585ed8d3bf0 tx=0x5585ecba6b40 comp rx=0 tx=0).stop 2024-09-17T04:34:23.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.921+0000 7ff76ad21280 1 -- 172.21.15.37:0/3726026532 shutdown_connections 2024-09-17T04:34:23.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.921+0000 7ff76ad21280 1 --2- 172.21.15.37:0/3726026532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5585ecb5a400 0x5585ed87ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.921+0000 7ff76ad21280 1 --2- 172.21.15.37:0/3726026532 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x5585ecb59b00 0x5585ed87a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.921+0000 7ff76ad21280 1 --2- 172.21.15.37:0/3726026532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585ecb59680 0x5585ed87a580 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:23.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.921+0000 7ff76ad21280 1 -- 172.21.15.37:0/3726026532 >> 172.21.15.37:0/3726026532 conn(0x5585ecb59f80 msgr2=0x5585ecbcd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:23.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.921+0000 7ff76ad21280 1 -- 172.21.15.37:0/3726026532 shutdown_connections 2024-09-17T04:34:23.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:23.921+0000 7ff76ad21280 1 -- 172.21.15.37:0/3726026532 wait complete. 2024-09-17T04:34:23.926 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'recovery_fs:0', 'journal', 'reset', '--force', '--yes-i-really-really-mean-it'] in time 0:00:00.084767, result: writing EResetJournal entry 2024-09-17T04:34:23.926 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set recovery_fs joinable true 2024-09-17T04:34:23.934 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-09-17T04:34:23.935+0000 7f4735687640 -1 log_channel(cluster) log [ERR] : Health check update: 2 filesystems are offline (MDS_ALL_DOWN) 2024-09-17T04:34:24.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.047+0000 7ff370230640 1 -- 172.21.15.37:0/1328966032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff354003b30 msgr2=0x7ff354003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:24.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.047+0000 7ff370230640 1 --2- 172.21.15.37:0/1328966032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff354003b30 0x7ff354003fa0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff364009b50 tx=0x7ff354005830 comp rx=0 tx=0).stop 2024-09-17T04:34:24.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.047+0000 7ff370230640 1 -- 172.21.15.37:0/1328966032 shutdown_connections 2024-09-17T04:34:24.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.047+0000 7ff370230640 1 --2- 172.21.15.37:0/1328966032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff354003b30 0x7ff354003fa0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:24.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.047+0000 7ff370230640 1 -- 172.21.15.37:0/1328966032 >> 172.21.15.37:0/1328966032 conn(0x7ff36808bef0 msgr2=0x7ff3680d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:24.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.047+0000 7ff370230640 1 -- 172.21.15.37:0/1328966032 shutdown_connections 2024-09-17T04:34:24.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.047+0000 7ff370230640 1 -- 172.21.15.37:0/1328966032 wait complete. 2024-09-17T04:34:24.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.049+0000 7ff370230640 1 Processor -- start 2024-09-17T04:34:24.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.049+0000 7ff370230640 1 -- start start 2024-09-17T04:34:24.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.050+0000 7ff370230640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff354003b30 0x7ff368157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:24.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.050+0000 7ff370230640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3680c93f0 0x7ff3680c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:24.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.050+0000 7ff370230640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3681e9fb0 0x7ff3681ea3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:24.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.050+0000 7ff370230640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff3680d4ec0 con 0x7ff3681e9fb0 2024-09-17T04:34:24.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.050+0000 7ff370230640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff3680d4d40 con 0x7ff3680c93f0 2024-09-17T04:34:24.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.050+0000 7ff370230640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7ff3680d5040 con 0x7ff354003b30 2024-09-17T04:34:24.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.051+0000 7ff36e7cd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3681e9fb0 0x7ff3681ea3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:24.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.051+0000 7ff36e7cd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3681e9fb0 0x7ff3681ea3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38906/0 (socket says 172.21.15.37:38906) 2024-09-17T04:34:24.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.051+0000 7ff36e7cd640 1 -- 172.21.15.37:0/2233403435 learned_addr learned my addr 172.21.15.37:0/2233403435 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:24.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.051+0000 7ff36dfcc640 1 --2- 172.21.15.37:0/2233403435 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff354003b30 0x7ff368157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:24.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.051+0000 7ff36d7cb640 1 --2- 172.21.15.37:0/2233403435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3680c93f0 0x7ff3680c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:24.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.051+0000 7ff36e7cd640 1 -- 172.21.15.37:0/2233403435 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff354003b30 msgr2=0x7ff368157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:24.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.051+0000 7ff36e7cd640 1 --2- 172.21.15.37:0/2233403435 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff354003b30 0x7ff368157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:24.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.051+0000 7ff36e7cd640 1 -- 172.21.15.37:0/2233403435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3680c93f0 msgr2=0x7ff3680c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:24.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.051+0000 7ff36e7cd640 1 --2- 172.21.15.37:0/2233403435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3680c93f0 0x7ff3680c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:24.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.051+0000 7ff36e7cd640 1 -- 172.21.15.37:0/2233403435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3681ea900 con 0x7ff3681e9fb0 2024-09-17T04:34:24.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.051+0000 7ff36d7cb640 1 --2- 172.21.15.37:0/2233403435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3680c93f0 0x7ff3680c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:24.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.051+0000 7ff36dfcc640 1 --2- 172.21.15.37:0/2233403435 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff354003b30 0x7ff368157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:24.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.051+0000 7ff36e7cd640 1 --2- 172.21.15.37:0/2233403435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3681e9fb0 0x7ff3681ea3c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff360000f80 tx=0x7ff36000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:24.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.051+0000 7ff35a7fc640 1 -- 172.21.15.37:0/2233403435 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36000c9b0 con 0x7ff3681e9fb0 2024-09-17T04:34:24.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.051+0000 7ff35a7fc640 1 -- 172.21.15.37:0/2233403435 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff360017070 con 0x7ff3681e9fb0 2024-09-17T04:34:24.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.051+0000 7ff370230640 1 -- 172.21.15.37:0/2233403435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3681f9ad0 con 0x7ff3681e9fb0 2024-09-17T04:34:24.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.051+0000 7ff35a7fc640 1 -- 172.21.15.37:0/2233403435 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff360011460 con 0x7ff3681e9fb0 2024-09-17T04:34:24.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.051+0000 7ff370230640 1 -- 172.21.15.37:0/2233403435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3681fa090 con 0x7ff3681e9fb0 2024-09-17T04:34:24.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.052+0000 7ff370230640 1 -- 172.21.15.37:0/2233403435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff368162c30 con 0x7ff3681e9fb0 2024-09-17T04:34:24.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.053+0000 7ff35a7fc640 1 -- 172.21.15.37:0/2233403435 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff360011600 con 0x7ff3681e9fb0 2024-09-17T04:34:24.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.054+0000 7ff35a7fc640 1 --2- 172.21.15.37:0/2233403435 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff34c078b90 0x7ff34c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:24.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.054+0000 7ff35a7fc640 1 -- 172.21.15.37:0/2233403435 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7ff36009b4a0 con 0x7ff3681e9fb0 2024-09-17T04:34:24.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.054+0000 7ff36dfcc640 1 --2- 172.21.15.37:0/2233403435 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff34c078b90 0x7ff34c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:24.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.054+0000 7ff36dfcc640 1 --2- 172.21.15.37:0/2233403435 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff34c078b90 0x7ff34c07b0a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff364035000 tx=0x7ff3640046c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:24.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.059+0000 7ff35a7fc640 1 -- 172.21.15.37:0/2233403435 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff3681ea900 con 0x7ff3681e9fb0 2024-09-17T04:34:24.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:24.287+0000 7ff370230640 1 -- 172.21.15.37:0/2233403435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "recovery_fs", "var": "joinable", "val": "true"} v 0) -- 0x7ff3680cfac0 con 0x7ff3681e9fb0 2024-09-17T04:34:25.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.230+0000 7ff35a7fc640 1 -- 172.21.15.37:0/2233403435 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "recovery_fs", "var": "joinable", "val": "true"}]=0 recovery_fs marked joinable; MDS may join as newly active. v23) ==== 172+0+0 (secure 0 0 0) 0x7ff360063620 con 0x7ff3681e9fb0 2024-09-17T04:34:25.230 INFO:teuthology.orchestra.run.smithi037.stderr:recovery_fs marked joinable; MDS may join as newly active. 2024-09-17T04:34:25.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.233+0000 7ff370230640 1 -- 172.21.15.37:0/2233403435 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff34c078b90 msgr2=0x7ff34c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:25.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.233+0000 7ff370230640 1 --2- 172.21.15.37:0/2233403435 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff34c078b90 0x7ff34c07b0a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff364035000 tx=0x7ff3640046c0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.233+0000 7ff370230640 1 -- 172.21.15.37:0/2233403435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3681e9fb0 msgr2=0x7ff3681ea3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:25.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.233+0000 7ff370230640 1 --2- 172.21.15.37:0/2233403435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3681e9fb0 0x7ff3681ea3c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff360000f80 tx=0x7ff36000eb80 comp rx=0 tx=0).stop 2024-09-17T04:34:25.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.233+0000 7ff370230640 1 -- 172.21.15.37:0/2233403435 shutdown_connections 2024-09-17T04:34:25.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.233+0000 7ff370230640 1 --2- 172.21.15.37:0/2233403435 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff34c078b90 0x7ff34c07b0a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.233+0000 7ff370230640 1 --2- 172.21.15.37:0/2233403435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3681e9fb0 0x7ff3681ea3c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.233+0000 7ff370230640 1 --2- 172.21.15.37:0/2233403435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3680c93f0 0x7ff3680c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.233+0000 7ff370230640 1 --2- 172.21.15.37:0/2233403435 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff354003b30 0x7ff368157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.233+0000 7ff370230640 1 -- 172.21.15.37:0/2233403435 >> 172.21.15.37:0/2233403435 conn(0x7ff36808bef0 msgr2=0x7ff3680cf2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:25.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.234+0000 7ff370230640 1 -- 172.21.15.37:0/2233403435 shutdown_connections 2024-09-17T04:34:25.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.234+0000 7ff370230640 1 -- 172.21.15.37:0/2233403435 wait complete. 2024-09-17T04:34:25.240 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-09-17T04:34:25.240+0000 7f55101d1640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2024-09-17T04:34:25.250 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:34:25.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.416+0000 7f63425c6640 1 -- 172.21.15.37:0/4061281716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6324005330 msgr2=0x7f632400b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:25.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.416+0000 7f63425c6640 1 --2- 172.21.15.37:0/4061281716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6324005330 0x7f632400b9e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f633400c770 tx=0x7f6324005830 comp rx=0 tx=0).stop 2024-09-17T04:34:25.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.416+0000 7f63425c6640 1 -- 172.21.15.37:0/4061281716 shutdown_connections 2024-09-17T04:34:25.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.416+0000 7f63425c6640 1 --2- 172.21.15.37:0/4061281716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6324005330 0x7f632400b9e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.416+0000 7f63425c6640 1 -- 172.21.15.37:0/4061281716 >> 172.21.15.37:0/4061281716 conn(0x7f633c08bef0 msgr2=0x7f633c0d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:25.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.416+0000 7f63425c6640 1 -- 172.21.15.37:0/4061281716 shutdown_connections 2024-09-17T04:34:25.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.417+0000 7f63425c6640 1 -- 172.21.15.37:0/4061281716 wait complete. 2024-09-17T04:34:25.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.418+0000 7f63425c6640 1 Processor -- start 2024-09-17T04:34:25.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.419+0000 7f63425c6640 1 -- start start 2024-09-17T04:34:25.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.420+0000 7f63425c6640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6324005330 0x7f633c15de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:25.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.420+0000 7f63425c6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f633c09fd00 0x7f633c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:25.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.420+0000 7f63425c6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f633c1e8d30 0x7f633c0b6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:25.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.420+0000 7f63425c6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f633c0b3980 con 0x7f633c09fd00 2024-09-17T04:34:25.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.420+0000 7f63425c6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f633c0d4d40 con 0x7f633c1e8d30 2024-09-17T04:34:25.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.420+0000 7f63425c6640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f633c0b3b00 con 0x7f6324005330 2024-09-17T04:34:25.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.420+0000 7f633b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f633c09fd00 0x7f633c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:25.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.420+0000 7f633b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f633c09fd00 0x7f633c0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48120/0 (socket says 172.21.15.37:48120) 2024-09-17T04:34:25.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.420+0000 7f633b7fe640 1 -- 172.21.15.37:0/448604178 learned_addr learned my addr 172.21.15.37:0/448604178 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:25.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.420+0000 7f633bfff640 1 --2- 172.21.15.37:0/448604178 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6324005330 0x7f633c15de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:25.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.420+0000 7f6340b63640 1 --2- 172.21.15.37:0/448604178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f633c1e8d30 0x7f633c0b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:25.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.420+0000 7f633b7fe640 1 -- 172.21.15.37:0/448604178 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6324005330 msgr2=0x7f633c15de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:25.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.420+0000 7f633b7fe640 1 --2- 172.21.15.37:0/448604178 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6324005330 0x7f633c15de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.420+0000 7f633b7fe640 1 -- 172.21.15.37:0/448604178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f633c1e8d30 msgr2=0x7f633c0b6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:25.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.420+0000 7f633b7fe640 1 --2- 172.21.15.37:0/448604178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f633c1e8d30 0x7f633c0b6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.420+0000 7f633b7fe640 1 -- 172.21.15.37:0/448604178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f633c15e720 con 0x7f633c09fd00 2024-09-17T04:34:25.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.420+0000 7f633bfff640 1 --2- 172.21.15.37:0/448604178 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6324005330 0x7f633c15de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:25.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.420+0000 7f6340b63640 1 --2- 172.21.15.37:0/448604178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f633c1e8d30 0x7f633c0b6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:25.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.420+0000 7f633b7fe640 1 --2- 172.21.15.37:0/448604178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f633c09fd00 0x7f633c0b5f40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6330003f10 tx=0x7f633000f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:25.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.421+0000 7f6338ff9640 1 -- 172.21.15.37:0/448604178 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6330015070 con 0x7f633c09fd00 2024-09-17T04:34:25.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.421+0000 7f6338ff9640 1 -- 172.21.15.37:0/448604178 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6330011860 con 0x7f633c09fd00 2024-09-17T04:34:25.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.421+0000 7f63425c6640 1 -- 172.21.15.37:0/448604178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f633c0a0da0 con 0x7f633c09fd00 2024-09-17T04:34:25.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.421+0000 7f6338ff9640 1 -- 172.21.15.37:0/448604178 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6330011cb0 con 0x7f633c09fd00 2024-09-17T04:34:25.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.421+0000 7f63425c6640 1 -- 172.21.15.37:0/448604178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f633c15ead0 con 0x7f633c09fd00 2024-09-17T04:34:25.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.422+0000 7f63425c6640 1 -- 172.21.15.37:0/448604178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f633c0d1070 con 0x7f633c09fd00 2024-09-17T04:34:25.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.422+0000 7f6338ff9640 1 -- 172.21.15.37:0/448604178 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f63300103c0 con 0x7f633c09fd00 2024-09-17T04:34:25.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.423+0000 7f6338ff9640 1 --2- 172.21.15.37:0/448604178 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6328078b90 0x7f632807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:25.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.423+0000 7f6338ff9640 1 -- 172.21.15.37:0/448604178 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7f633009ba30 con 0x7f633c09fd00 2024-09-17T04:34:25.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.424+0000 7f633bfff640 1 --2- 172.21.15.37:0/448604178 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6328078b90 0x7f632807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:25.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.424+0000 7f633bfff640 1 --2- 172.21.15.37:0/448604178 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6328078b90 0x7f632807b0a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f632c009bd0 tx=0x7f632c005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:25.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.428+0000 7f6338ff9640 1 -- 172.21.15.37:0/448604178 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6330063ce0 con 0x7f633c09fd00 2024-09-17T04:34:25.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.652+0000 7f63425c6640 1 -- 172.21.15.37:0/448604178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f633c06a150 con 0x7f633c09fd00 2024-09-17T04:34:25.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.652+0000 7f6338ff9640 1 -- 172.21.15.37:0/448604178 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+6515 (secure 0 0 0) 0x7f63300635e0 con 0x7f633c09fd00 2024-09-17T04:34:25.652 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:25.652 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":24,"btime":"2024-09-17T04:34:25:234339+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/199192833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":199192833},{"type":"v1","addr":"172.21.15.37:6833","nonce":199192833}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4832,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/4098913371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4098913371},{"type":"v1","addr":"172.21.15.92:6836","nonce":4098913371}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/383658044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":383658044},{"type":"v1","addr":"172.21.15.92:6837","nonce":383658044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:59.490265+0000","modified":"2024-09-17T04:34:15.700096+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2},{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:34:17.928780+0000","modified":"2024-09-17T04:34:25.234323+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5208":{"gid":5208,"name":"a","rank":0,"incarnation":24,"state":"up:replay","state_seq":1,"addr":"172.21.15.37:6835/4051839052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":4051839052},{"type":"v1","addr":"172.21.15.37:6835","nonce":4051839052}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":6,"enabled":true,"fs_name":"recovery_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-09-17T04:34:25.653 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 24 2024-09-17T04:34:25.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.655+0000 7f63425c6640 1 -- 172.21.15.37:0/448604178 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6328078b90 msgr2=0x7f632807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:25.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.655+0000 7f63425c6640 1 --2- 172.21.15.37:0/448604178 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6328078b90 0x7f632807b0a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f632c009bd0 tx=0x7f632c005f90 comp rx=0 tx=0).stop 2024-09-17T04:34:25.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.655+0000 7f63425c6640 1 -- 172.21.15.37:0/448604178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f633c09fd00 msgr2=0x7f633c0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:25.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.655+0000 7f63425c6640 1 --2- 172.21.15.37:0/448604178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f633c09fd00 0x7f633c0b5f40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6330003f10 tx=0x7f633000f720 comp rx=0 tx=0).stop 2024-09-17T04:34:25.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.655+0000 7f63425c6640 1 -- 172.21.15.37:0/448604178 shutdown_connections 2024-09-17T04:34:25.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.655+0000 7f63425c6640 1 --2- 172.21.15.37:0/448604178 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6328078b90 0x7f632807b0a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.655+0000 7f63425c6640 1 --2- 172.21.15.37:0/448604178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f633c1e8d30 0x7f633c0b6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.655+0000 7f63425c6640 1 --2- 172.21.15.37:0/448604178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f633c09fd00 0x7f633c0b5f40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.655+0000 7f63425c6640 1 --2- 172.21.15.37:0/448604178 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6324005330 0x7f633c15de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:25.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.655+0000 7f63425c6640 1 -- 172.21.15.37:0/448604178 >> 172.21.15.37:0/448604178 conn(0x7f633c08bef0 msgr2=0x7f633c0b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:25.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.655+0000 7f63425c6640 1 -- 172.21.15.37:0/448604178 shutdown_connections 2024-09-17T04:34:25.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:25.655+0000 7f63425c6640 1 -- 172.21.15.37:0/448604178 wait complete. 2024-09-17T04:34:26.069 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T04:34:17.928780+0000', 'modified': '2024-09-17T04:34:25.234323+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5208}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5208': {'gid': 5208, 'name': 'a', 'rank': 0, 'incarnation': 24, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.37:6835/4051839052', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 4051839052}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 4051839052}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'recovery_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T04:34:26.069 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5208:up:replay 2024-09-17T04:34:27.070 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:34:27.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.202+0000 7fe035412640 1 -- 172.21.15.37:0/2207612643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0300d1610 msgr2=0x7fe0300d23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:27.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.202+0000 7fe035412640 1 --2- 172.21.15.37:0/2207612643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0300d1610 0x7fe0300d23b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe0300cb6b0 tx=0x7fe02402a510 comp rx=0 tx=0).stop 2024-09-17T04:34:27.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.202+0000 7fe035412640 1 -- 172.21.15.37:0/2207612643 shutdown_connections 2024-09-17T04:34:27.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.202+0000 7fe035412640 1 --2- 172.21.15.37:0/2207612643 >> v2:172.21.15.92:3300/0 conn(0x7fe0300b3430 0x7fe0300d28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:27.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.202+0000 7fe035412640 1 --2- 172.21.15.37:0/2207612643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0300d1610 0x7fe0300d23b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:27.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.202+0000 7fe035412640 1 -- 172.21.15.37:0/2207612643 >> 172.21.15.37:0/2207612643 conn(0x7fe03008bef0 msgr2=0x7fe0300d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:27.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.202+0000 7fe035412640 1 -- 172.21.15.37:0/2207612643 shutdown_connections 2024-09-17T04:34:27.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.202+0000 7fe035412640 1 -- 172.21.15.37:0/2207612643 wait complete. 2024-09-17T04:34:27.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.204+0000 7fe035412640 1 Processor -- start 2024-09-17T04:34:27.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.204+0000 7fe035412640 1 -- start start 2024-09-17T04:34:27.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.206+0000 7fe035412640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe03006f9f0 0x7fe03015bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:27.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.206+0000 7fe035412640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe0300b3430 0x7fe0300b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:27.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.206+0000 7fe035412640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0300b5620 0x7fe0300b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:27.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.206+0000 7fe035412640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe0300d2e30 con 0x7fe03006f9f0 2024-09-17T04:34:27.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.206+0000 7fe035412640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe0300b3b90 con 0x7fe0300b5620 2024-09-17T04:34:27.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.206+0000 7fe035412640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fe0300d2fb0 con 0x7fe0300b3430 2024-09-17T04:34:27.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.206+0000 7fe02effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe03006f9f0 0x7fe03015bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:27.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.206+0000 7fe02effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe03006f9f0 0x7fe03015bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48142/0 (socket says 172.21.15.37:48142) 2024-09-17T04:34:27.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.206+0000 7fe02effd640 1 -- 172.21.15.37:0/3902770973 learned_addr learned my addr 172.21.15.37:0/3902770973 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:27.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.206+0000 7fe02f7fe640 1 --2- 172.21.15.37:0/3902770973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0300b5620 0x7fe0300b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:27.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.206+0000 7fe02e7fc640 1 --2- 172.21.15.37:0/3902770973 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe0300b3430 0x7fe0300b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:27.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.207+0000 7fe02effd640 1 -- 172.21.15.37:0/3902770973 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe0300b3430 msgr2=0x7fe0300b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:27.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.207+0000 7fe02effd640 1 --2- 172.21.15.37:0/3902770973 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe0300b3430 0x7fe0300b50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:27.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.207+0000 7fe02effd640 1 -- 172.21.15.37:0/3902770973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0300b5620 msgr2=0x7fe0300b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:27.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.207+0000 7fe02effd640 1 --2- 172.21.15.37:0/3902770973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0300b5620 0x7fe0300b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:27.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.207+0000 7fe02effd640 1 -- 172.21.15.37:0/3902770973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe03015c2c0 con 0x7fe03006f9f0 2024-09-17T04:34:27.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.207+0000 7fe02effd640 1 --2- 172.21.15.37:0/3902770973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe03006f9f0 0x7fe03015bd80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe02000b420 tx=0x7fe02000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:27.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.207+0000 7fe0177fe640 1 -- 172.21.15.37:0/3902770973 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020011830 con 0x7fe03006f9f0 2024-09-17T04:34:27.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.207+0000 7fe0177fe640 1 -- 172.21.15.37:0/3902770973 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe020016070 con 0x7fe03006f9f0 2024-09-17T04:34:27.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.207+0000 7fe035412640 1 -- 172.21.15.37:0/3902770973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe03015dca0 con 0x7fe03006f9f0 2024-09-17T04:34:27.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.207+0000 7fe0177fe640 1 -- 172.21.15.37:0/3902770973 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020011d60 con 0x7fe03006f9f0 2024-09-17T04:34:27.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.207+0000 7fe035412640 1 -- 172.21.15.37:0/3902770973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe03015e260 con 0x7fe03006f9f0 2024-09-17T04:34:27.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.208+0000 7fe035412640 1 -- 172.21.15.37:0/3902770973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0300d3460 con 0x7fe03006f9f0 2024-09-17T04:34:27.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.209+0000 7fe0177fe640 1 -- 172.21.15.37:0/3902770973 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe02001b460 con 0x7fe03006f9f0 2024-09-17T04:34:27.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.209+0000 7fe0177fe640 1 --2- 172.21.15.37:0/3902770973 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe01c078b90 0x7fe01c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:27.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.210+0000 7fe0177fe640 1 -- 172.21.15.37:0/3902770973 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7fe02009abd0 con 0x7fe03006f9f0 2024-09-17T04:34:27.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.210+0000 7fe02e7fc640 1 --2- 172.21.15.37:0/3902770973 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe01c078b90 0x7fe01c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:27.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.210+0000 7fe02e7fc640 1 --2- 172.21.15.37:0/3902770973 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe01c078b90 0x7fe01c07b0a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe024034000 tx=0x7fe024004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:27.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.214+0000 7fe0177fe640 1 -- 172.21.15.37:0/3902770973 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe020062e80 con 0x7fe03006f9f0 2024-09-17T04:34:27.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.440+0000 7fe035412640 1 -- 172.21.15.37:0/3902770973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe03006a150 con 0x7fe03006f9f0 2024-09-17T04:34:27.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.440+0000 7fe0177fe640 1 -- 172.21.15.37:0/3902770973 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+6515 (secure 0 0 0) 0x7fe020062780 con 0x7fe03006f9f0 2024-09-17T04:34:27.440 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:27.441 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"btime":"2024-09-17T04:34:27:238671+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/199192833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":199192833},{"type":"v1","addr":"172.21.15.37:6833","nonce":199192833}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4832,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/4098913371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4098913371},{"type":"v1","addr":"172.21.15.92:6836","nonce":4098913371}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/383658044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":383658044},{"type":"v1","addr":"172.21.15.92:6837","nonce":383658044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:59.490265+0000","modified":"2024-09-17T04:34:15.700096+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2},{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:34:17.928780+0000","modified":"2024-09-17T04:34:26.240445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5208":{"gid":5208,"name":"a","rank":0,"incarnation":24,"state":"up:rejoin","state_seq":5,"addr":"172.21.15.37:6835/4051839052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":4051839052},{"type":"v1","addr":"172.21.15.37:6835","nonce":4051839052}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":6,"enabled":true,"fs_name":"recovery_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-09-17T04:34:27.441 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-09-17T04:34:27.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.443+0000 7fe035412640 1 -- 172.21.15.37:0/3902770973 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe01c078b90 msgr2=0x7fe01c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:27.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.443+0000 7fe035412640 1 --2- 172.21.15.37:0/3902770973 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe01c078b90 0x7fe01c07b0a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe024034000 tx=0x7fe024004730 comp rx=0 tx=0).stop 2024-09-17T04:34:27.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.443+0000 7fe035412640 1 -- 172.21.15.37:0/3902770973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe03006f9f0 msgr2=0x7fe03015bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:27.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.443+0000 7fe035412640 1 --2- 172.21.15.37:0/3902770973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe03006f9f0 0x7fe03015bd80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe02000b420 tx=0x7fe02000b8e0 comp rx=0 tx=0).stop 2024-09-17T04:34:27.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.443+0000 7fe035412640 1 -- 172.21.15.37:0/3902770973 shutdown_connections 2024-09-17T04:34:27.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.443+0000 7fe035412640 1 --2- 172.21.15.37:0/3902770973 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe01c078b90 0x7fe01c07b0a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:27.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.443+0000 7fe035412640 1 --2- 172.21.15.37:0/3902770973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0300b5620 0x7fe0300b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:27.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.443+0000 7fe035412640 1 --2- 172.21.15.37:0/3902770973 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe0300b3430 0x7fe0300b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:27.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.443+0000 7fe035412640 1 --2- 172.21.15.37:0/3902770973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe03006f9f0 0x7fe03015bd80 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:27.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.443+0000 7fe035412640 1 -- 172.21.15.37:0/3902770973 >> 172.21.15.37:0/3902770973 conn(0x7fe03008bef0 msgr2=0x7fe0300d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:27.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.443+0000 7fe035412640 1 -- 172.21.15.37:0/3902770973 shutdown_connections 2024-09-17T04:34:27.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:27.443+0000 7fe035412640 1 -- 172.21.15.37:0/3902770973 wait complete. 2024-09-17T04:34:27.857 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T04:34:17.928780+0000', 'modified': '2024-09-17T04:34:26.240445+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5208}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5208': {'gid': 5208, 'name': 'a', 'rank': 0, 'incarnation': 24, 'state': 'up:rejoin', 'state_seq': 5, 'addr': '172.21.15.37:6835/4051839052', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 4051839052}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 4051839052}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'recovery_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T04:34:27.857 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5208:up:rejoin 2024-09-17T04:34:28.858 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:34:28.935 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-09-17T04:34:28.935+0000 7f4735687640 -1 log_channel(cluster) log [ERR] : Health check update: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T04:34:28.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.992+0000 7f86d3c25640 1 -- 172.21.15.37:0/3264294435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86b8003ae0 msgr2=0x7f86b8003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:28.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.992+0000 7f86d3c25640 1 --2- 172.21.15.37:0/3264294435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86b8003ae0 0x7f86b8003f50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f86c8009b20 tx=0x7f86b8005830 comp rx=0 tx=0).stop 2024-09-17T04:34:28.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.992+0000 7f86d21c2640 1 -- 172.21.15.37:0/3264294435 reap_dead start 2024-09-17T04:34:28.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.992+0000 7f86d3c25640 1 -- 172.21.15.37:0/3264294435 shutdown_connections 2024-09-17T04:34:28.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.992+0000 7f86d3c25640 1 -- 172.21.15.37:0/3264294435 >> 172.21.15.37:0/3264294435 conn(0x7f86cc0d1330 msgr2=0x7f86cc0d1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:28.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.992+0000 7f86d3c25640 1 -- 172.21.15.37:0/3264294435 shutdown_connections 2024-09-17T04:34:28.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.992+0000 7f86d3c25640 1 -- 172.21.15.37:0/3264294435 wait complete. 2024-09-17T04:34:28.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.994+0000 7f86d3c25640 1 Processor -- start 2024-09-17T04:34:28.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.994+0000 7f86d3c25640 1 -- start start 2024-09-17T04:34:28.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.995+0000 7f86d3c25640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86b8003ae0 0x7f86cc1591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:28.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.995+0000 7f86d3c25640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f86cc1f52a0 0x7f86cc070e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:28.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.995+0000 7f86d3c25640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86cc071380 0x7f86cc161580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:28.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.995+0000 7f86d3c25640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f86cc0b14e0 con 0x7f86b8003ae0 2024-09-17T04:34:28.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.995+0000 7f86d3c25640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f86cc0ce8c0 con 0x7f86cc071380 2024-09-17T04:34:28.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.995+0000 7f86d3c25640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f86cc0b1660 con 0x7f86cc1f52a0 2024-09-17T04:34:28.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.995+0000 7f86d19c1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86b8003ae0 0x7f86cc1591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:28.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.995+0000 7f86d19c1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86b8003ae0 0x7f86cc1591c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48158/0 (socket says 172.21.15.37:48158) 2024-09-17T04:34:28.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.995+0000 7f86d19c1640 1 -- 172.21.15.37:0/18021024 learned_addr learned my addr 172.21.15.37:0/18021024 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:28.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.995+0000 7f86d11c0640 1 --2- 172.21.15.37:0/18021024 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f86cc1f52a0 0x7f86cc070e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:28.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.995+0000 7f86d21c2640 1 --2- 172.21.15.37:0/18021024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86cc071380 0x7f86cc161580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:28.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.996+0000 7f86d19c1640 1 -- 172.21.15.37:0/18021024 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f86cc1f52a0 msgr2=0x7f86cc070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:28.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.996+0000 7f86d19c1640 1 --2- 172.21.15.37:0/18021024 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f86cc1f52a0 0x7f86cc070e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:28.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.996+0000 7f86d19c1640 1 -- 172.21.15.37:0/18021024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86cc071380 msgr2=0x7f86cc161580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:28.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.996+0000 7f86d19c1640 1 --2- 172.21.15.37:0/18021024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86cc071380 0x7f86cc161580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:28.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.996+0000 7f86d19c1640 1 -- 172.21.15.37:0/18021024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86cc15ae10 con 0x7f86b8003ae0 2024-09-17T04:34:28.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.996+0000 7f86d19c1640 1 --2- 172.21.15.37:0/18021024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86b8003ae0 0x7f86cc1591c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f86c8009b20 tx=0x7f86c8033650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:28.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.996+0000 7f86c27fc640 1 -- 172.21.15.37:0/18021024 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8038040 con 0x7f86b8003ae0 2024-09-17T04:34:28.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.996+0000 7f86c27fc640 1 -- 172.21.15.37:0/18021024 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86c8033840 con 0x7f86b8003ae0 2024-09-17T04:34:28.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.996+0000 7f86c27fc640 1 -- 172.21.15.37:0/18021024 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8033c90 con 0x7f86b8003ae0 2024-09-17T04:34:28.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.996+0000 7f86d3c25640 1 -- 172.21.15.37:0/18021024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86cc071790 con 0x7f86b8003ae0 2024-09-17T04:34:28.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.996+0000 7f86d3c25640 1 -- 172.21.15.37:0/18021024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86cc1f5f60 con 0x7f86b8003ae0 2024-09-17T04:34:28.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.997+0000 7f86d3c25640 1 -- 172.21.15.37:0/18021024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86cc1f61a0 con 0x7f86b8003ae0 2024-09-17T04:34:28.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.998+0000 7f86c27fc640 1 -- 172.21.15.37:0/18021024 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f86c8045020 con 0x7f86b8003ae0 2024-09-17T04:34:28.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.998+0000 7f86c27fc640 1 --2- 172.21.15.37:0/18021024 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f86b0078b90 0x7f86b007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:28.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.998+0000 7f86c27fc640 1 -- 172.21.15.37:0/18021024 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7f86c80bb920 con 0x7f86b8003ae0 2024-09-17T04:34:28.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.999+0000 7f86d11c0640 1 --2- 172.21.15.37:0/18021024 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f86b0078b90 0x7f86b007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:28.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:28.999+0000 7f86d11c0640 1 --2- 172.21.15.37:0/18021024 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f86b0078b90 0x7f86b007b0a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0b06d0 tx=0x7f86bc005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:29.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.003+0000 7f86c27fc640 1 -- 172.21.15.37:0/18021024 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f86cc1f61a0 con 0x7f86b8003ae0 2024-09-17T04:34:29.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.227+0000 7f86d3c25640 1 -- 172.21.15.37:0/18021024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86cc0b0f70 con 0x7f86b8003ae0 2024-09-17T04:34:29.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.227+0000 7f86c27fc640 1 -- 172.21.15.37:0/18021024 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+6522 (secure 0 0 0) 0x7f86cc071790 con 0x7f86b8003ae0 2024-09-17T04:34:29.227 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:29.228 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"btime":"2024-09-17T04:34:28:244441+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/199192833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":199192833},{"type":"v1","addr":"172.21.15.37:6833","nonce":199192833}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4832,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/4098913371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4098913371},{"type":"v1","addr":"172.21.15.92:6836","nonce":4098913371}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/383658044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":383658044},{"type":"v1","addr":"172.21.15.92:6837","nonce":383658044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:59.490265+0000","modified":"2024-09-17T04:34:15.700096+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2},{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:34:17.928780+0000","modified":"2024-09-17T04:34:28.244429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5208":{"gid":5208,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6835/4051839052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":4051839052},{"type":"v1","addr":"172.21.15.37:6835","nonce":4051839052}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":6,"enabled":true,"fs_name":"recovery_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5208,"qdb_cluster":[5208]},"id":3}]} 2024-09-17T04:34:29.228 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-09-17T04:34:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.230+0000 7f86d3c25640 1 -- 172.21.15.37:0/18021024 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f86b0078b90 msgr2=0x7f86b007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.230+0000 7f86d3c25640 1 --2- 172.21.15.37:0/18021024 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f86b0078b90 0x7f86b007b0a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0b06d0 tx=0x7f86bc005f90 comp rx=0 tx=0).stop 2024-09-17T04:34:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.230+0000 7f86d3c25640 1 -- 172.21.15.37:0/18021024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86b8003ae0 msgr2=0x7f86cc1591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.230+0000 7f86d3c25640 1 --2- 172.21.15.37:0/18021024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86b8003ae0 0x7f86cc1591c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f86c8009b20 tx=0x7f86c8033650 comp rx=0 tx=0).stop 2024-09-17T04:34:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.230+0000 7f86d3c25640 1 -- 172.21.15.37:0/18021024 shutdown_connections 2024-09-17T04:34:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.230+0000 7f86d3c25640 1 --2- 172.21.15.37:0/18021024 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f86b0078b90 0x7f86b007b0a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.230+0000 7f86d3c25640 1 --2- 172.21.15.37:0/18021024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86cc071380 0x7f86cc161580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.230+0000 7f86d3c25640 1 --2- 172.21.15.37:0/18021024 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f86cc1f52a0 0x7f86cc070e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.230+0000 7f86d3c25640 1 --2- 172.21.15.37:0/18021024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86b8003ae0 0x7f86cc1591c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.230+0000 7f86d3c25640 1 -- 172.21.15.37:0/18021024 >> 172.21.15.37:0/18021024 conn(0x7f86cc0d1330 msgr2=0x7f86cc0b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:29.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.230+0000 7f86d3c25640 1 -- 172.21.15.37:0/18021024 shutdown_connections 2024-09-17T04:34:29.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.230+0000 7f86d3c25640 1 -- 172.21.15.37:0/18021024 wait complete. 2024-09-17T04:34:29.643 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T04:34:17.928780+0000', 'modified': '2024-09-17T04:34:28.244429+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5208}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5208': {'gid': 5208, 'name': 'a', 'rank': 0, 'incarnation': 24, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.37:6835/4051839052', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 4051839052}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 4051839052}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'recovery_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5208, 'qdb_cluster': [5208]} 2024-09-17T04:34:29.644 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T04:34:29.644 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-17T04:34:29.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.766+0000 7ff5278cc640 1 -- 172.21.15.37:0/4032274204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff50c004d40 msgr2=0x7ff50c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:29.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.766+0000 7ff5278cc640 1 --2- 172.21.15.37:0/4032274204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff50c004d40 0x7ff50c005150 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff5200b33e0 tx=0x7ff50c005830 comp rx=0 tx=0).stop 2024-09-17T04:34:29.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.766+0000 7ff525e69640 1 -- 172.21.15.37:0/4032274204 reap_dead start 2024-09-17T04:34:29.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.766+0000 7ff5278cc640 1 -- 172.21.15.37:0/4032274204 shutdown_connections 2024-09-17T04:34:29.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.766+0000 7ff5278cc640 1 -- 172.21.15.37:0/4032274204 >> 172.21.15.37:0/4032274204 conn(0x7ff52008bef0 msgr2=0x7ff5200d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:29.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.767+0000 7ff5278cc640 1 -- 172.21.15.37:0/4032274204 shutdown_connections 2024-09-17T04:34:29.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.767+0000 7ff5278cc640 1 -- 172.21.15.37:0/4032274204 wait complete. 2024-09-17T04:34:29.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.768+0000 7ff5278cc640 1 Processor -- start 2024-09-17T04:34:29.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.768+0000 7ff5278cc640 1 -- start start 2024-09-17T04:34:29.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.770+0000 7ff5278cc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff50c004d40 0x7ff520159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:29.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.770+0000 7ff5278cc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff52014e250 0x7ff5201f8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:29.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.770+0000 7ff5278cc640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff5201f8ca0 0x7ff5201f90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:29.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.770+0000 7ff5278cc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff5200d4e20 con 0x7ff52014e250 2024-09-17T04:34:29.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.770+0000 7ff5278cc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff5200b3b90 con 0x7ff50c004d40 2024-09-17T04:34:29.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.770+0000 7ff5278cc640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7ff5200d28f0 con 0x7ff5201f8ca0 2024-09-17T04:34:29.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.770+0000 7ff524e67640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff52014e250 0x7ff5201f8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:29.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.770+0000 7ff524e67640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff52014e250 0x7ff5201f8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48176/0 (socket says 172.21.15.37:48176) 2024-09-17T04:34:29.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.770+0000 7ff524e67640 1 -- 172.21.15.37:0/3775407912 learned_addr learned my addr 172.21.15.37:0/3775407912 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:29.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.770+0000 7ff525668640 1 --2- 172.21.15.37:0/3775407912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff50c004d40 0x7ff520159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:29.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.770+0000 7ff525e69640 1 --2- 172.21.15.37:0/3775407912 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff5201f8ca0 0x7ff5201f90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:29.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.770+0000 7ff524e67640 1 -- 172.21.15.37:0/3775407912 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff5201f8ca0 msgr2=0x7ff5201f90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:29.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.770+0000 7ff524e67640 1 --2- 172.21.15.37:0/3775407912 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff5201f8ca0 0x7ff5201f90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.770+0000 7ff524e67640 1 -- 172.21.15.37:0/3775407912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff50c004d40 msgr2=0x7ff520159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:29.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.770+0000 7ff524e67640 1 --2- 172.21.15.37:0/3775407912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff50c004d40 0x7ff520159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.770+0000 7ff524e67640 1 -- 172.21.15.37:0/3775407912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5200cba10 con 0x7ff52014e250 2024-09-17T04:34:29.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.770+0000 7ff525e69640 1 --2- 172.21.15.37:0/3775407912 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff5201f8ca0 0x7ff5201f90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:29.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.770+0000 7ff524e67640 1 --2- 172.21.15.37:0/3775407912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff52014e250 0x7ff5201f8760 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff510011ae0 tx=0x7ff510009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:29.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.771+0000 7ff515ffb640 1 -- 172.21.15.37:0/3775407912 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51001e070 con 0x7ff52014e250 2024-09-17T04:34:29.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.771+0000 7ff515ffb640 1 -- 172.21.15.37:0/3775407912 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff51000fc70 con 0x7ff52014e250 2024-09-17T04:34:29.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.771+0000 7ff5278cc640 1 -- 172.21.15.37:0/3775407912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5200b6c90 con 0x7ff52014e250 2024-09-17T04:34:29.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.771+0000 7ff515ffb640 1 -- 172.21.15.37:0/3775407912 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510019410 con 0x7ff52014e250 2024-09-17T04:34:29.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.771+0000 7ff5278cc640 1 -- 172.21.15.37:0/3775407912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5200cd030 con 0x7ff52014e250 2024-09-17T04:34:29.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.773+0000 7ff515ffb640 1 -- 172.21.15.37:0/3775407912 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff510021070 con 0x7ff52014e250 2024-09-17T04:34:29.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.773+0000 7ff515ffb640 1 --2- 172.21.15.37:0/3775407912 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff504078b90 0x7ff50407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:29.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.773+0000 7ff515ffb640 1 -- 172.21.15.37:0/3775407912 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7ff51009b880 con 0x7ff52014e250 2024-09-17T04:34:29.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.774+0000 7ff525668640 1 --2- 172.21.15.37:0/3775407912 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff504078b90 0x7ff50407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:29.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.774+0000 7ff52e838740 1 Processor -- start 2024-09-17T04:34:29.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.774+0000 7ff52e838740 1 -- start start 2024-09-17T04:34:29.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.774+0000 7ff525668640 1 --2- 172.21.15.37:0/3775407912 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff504078b90 0x7ff50407b0a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff51c001c20 tx=0x7ff51c001bb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:29.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.775+0000 7ff52e838740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ed6243270 0x559ed6614fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:29.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.775+0000 7ff52e838740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed621a3b0 0x559ed66154e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:29.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.775+0000 7ff52e838740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559ed6222590 0x559ed66323e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:29.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.775+0000 7ff52e838740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559ed655d820 con 0x559ed621a3b0 2024-09-17T04:34:29.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.775+0000 7ff52e838740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559ed654cc50 con 0x559ed6243270 2024-09-17T04:34:29.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.775+0000 7ff52e838740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x559ed6550a60 con 0x559ed6222590 2024-09-17T04:34:29.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.775+0000 7ff524e67640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed621a3b0 0x559ed66154e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:29.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.776+0000 7ff524e67640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed621a3b0 0x559ed66154e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48186/0 (socket says 172.21.15.37:48186) 2024-09-17T04:34:29.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.776+0000 7ff524e67640 1 -- 172.21.15.37:0/53058449 learned_addr learned my addr 172.21.15.37:0/53058449 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:29.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.776+0000 7ff525668640 1 --2- 172.21.15.37:0/53058449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ed6243270 0x559ed6614fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:29.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.776+0000 7ff525e69640 1 --2- 172.21.15.37:0/53058449 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559ed6222590 0x559ed66323e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:29.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.776+0000 7ff524e67640 1 -- 172.21.15.37:0/53058449 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559ed6222590 msgr2=0x559ed66323e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:29.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.776+0000 7ff524e67640 1 --2- 172.21.15.37:0/53058449 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559ed6222590 0x559ed66323e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.776+0000 7ff524e67640 1 -- 172.21.15.37:0/53058449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ed6243270 msgr2=0x559ed6614fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:29.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.776+0000 7ff524e67640 1 --2- 172.21.15.37:0/53058449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ed6243270 0x559ed6614fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.776+0000 7ff524e67640 1 -- 172.21.15.37:0/53058449 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ed62ae850 con 0x559ed621a3b0 2024-09-17T04:34:29.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.776+0000 7ff524e67640 1 --2- 172.21.15.37:0/53058449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed621a3b0 0x559ed66154e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff510012fd0 tx=0x7ff510062bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666468b8fa9ea1c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:29.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.776+0000 7ff50affd640 1 -- 172.21.15.37:0/53058449 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510067e70 con 0x559ed621a3b0 2024-09-17T04:34:29.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.776+0000 7ff50affd640 1 -- 172.21.15.37:0/53058449 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff51006cd20 con 0x559ed621a3b0 2024-09-17T04:34:29.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.776+0000 7ff50affd640 1 -- 172.21.15.37:0/53058449 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51006a320 con 0x559ed621a3b0 2024-09-17T04:34:29.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.777+0000 7ff52e838740 1 -- 172.21.15.37:0/53058449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed621a3b0 msgr2=0x559ed66154e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:29.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.777+0000 7ff52e838740 1 --2- 172.21.15.37:0/53058449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed621a3b0 0x559ed66154e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff510012fd0 tx=0x7ff510062bb0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.777+0000 7ff52e838740 1 -- 172.21.15.37:0/53058449 shutdown_connections 2024-09-17T04:34:29.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.777+0000 7ff52e838740 1 --2- 172.21.15.37:0/53058449 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559ed6222590 0x559ed66323e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.777+0000 7ff52e838740 1 --2- 172.21.15.37:0/53058449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed621a3b0 0x559ed66154e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.777+0000 7ff52e838740 1 --2- 172.21.15.37:0/53058449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ed6243270 0x559ed6614fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.777+0000 7ff52e838740 1 -- 172.21.15.37:0/53058449 >> 172.21.15.37:0/53058449 conn(0x559ed62373c0 msgr2=0x559ed65eb260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:29.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.777+0000 7ff52e838740 1 -- 172.21.15.37:0/53058449 shutdown_connections 2024-09-17T04:34:29.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.777+0000 7ff52e838740 1 -- 172.21.15.37:0/53058449 wait complete. 2024-09-17T04:34:29.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.778+0000 7ff52e838740 1 -- start start 2024-09-17T04:34:29.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.778+0000 7ff52e838740 1 Processor -- start 2024-09-17T04:34:29.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.779+0000 7ff52e838740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed621a3b0 0x559ed66435b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:29.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.779+0000 7ff52e838740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559ed6222590 0x559ed663ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:29.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.779+0000 7ff52e838740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ed6243270 0x559ed65fe290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:29.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.779+0000 7ff52e838740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559ed62bd620 con 0x559ed621a3b0 2024-09-17T04:34:29.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.779+0000 7ff52e838740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559ed64f25f0 con 0x559ed6243270 2024-09-17T04:34:29.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.779+0000 7ff52e838740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x559ed6615a20 con 0x559ed6222590 2024-09-17T04:34:29.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.780+0000 7ff52e838740 10 client.0 _ll_register_callbacks cb 0x559ed64f60a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:34:29.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.780+0000 7ff525668640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed621a3b0 0x559ed66435b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:29.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.780+0000 7ff525668640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed621a3b0 0x559ed66435b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48202/0 (socket says 172.21.15.37:48202) 2024-09-17T04:34:29.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.780+0000 7ff525668640 1 -- 172.21.15.37:0/3784590401 learned_addr learned my addr 172.21.15.37:0/3784590401 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:29.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.780+0000 7ff525e69640 1 --2- 172.21.15.37:0/3784590401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ed6243270 0x559ed65fe290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:29.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.780+0000 7ff524e67640 1 --2- 172.21.15.37:0/3784590401 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559ed6222590 0x559ed663ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:29.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.780+0000 7ff525668640 1 -- 172.21.15.37:0/3784590401 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559ed6222590 msgr2=0x559ed663ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:29.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.780+0000 7ff525668640 1 --2- 172.21.15.37:0/3784590401 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559ed6222590 0x559ed663ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.780+0000 7ff525668640 1 -- 172.21.15.37:0/3784590401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ed6243270 msgr2=0x559ed65fe290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:29.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.780+0000 7ff525668640 1 --2- 172.21.15.37:0/3784590401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ed6243270 0x559ed65fe290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.780+0000 7ff525668640 1 -- 172.21.15.37:0/3784590401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ed63256b0 con 0x559ed621a3b0 2024-09-17T04:34:29.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.780+0000 7ff525668640 1 --2- 172.21.15.37:0/3784590401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed621a3b0 0x559ed66435b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff51c012440 tx=0x7ff5200d5280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a215a57fee07813 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:29.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.780+0000 7ff4fffff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-17T04:34:29.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.780+0000 7ff4fffff640 1 -- 172.21.15.37:0/3784590401 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c01f610 con 0x559ed621a3b0 2024-09-17T04:34:29.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.780+0000 7ff4fffff640 1 -- 172.21.15.37:0/3784590401 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7ff51c018b60 con 0x559ed621a3b0 2024-09-17T04:34:29.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.780+0000 7ff52e838740 1 -- 172.21.15.37:0/3784590401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff51006cd20 con 0x559ed621a3b0 2024-09-17T04:34:29.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.780+0000 7ff4fffff640 1 -- 172.21.15.37:0/3784590401 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff51c030050 con 0x559ed621a3b0 2024-09-17T04:34:29.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.780+0000 7ff52e838740 10 client.5433 fetch_fsmap learned FSMap version 27 2024-09-17T04:34:29.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.780+0000 7ff52e838740 1 -- 172.21.15.37:0/3784590401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x559ed6642e10 con 0x559ed621a3b0 2024-09-17T04:34:29.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.781+0000 7ff4fffff640 1 -- 172.21.15.37:0/3784590401 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff51006cd20 con 0x559ed621a3b0 2024-09-17T04:34:29.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.781+0000 7ff4fffff640 1 -- 172.21.15.37:0/3784590401 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 4147+0+0 (secure 0 0 0) 0x7ff51c033020 con 0x559ed621a3b0 2024-09-17T04:34:29.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.781+0000 7ff52e838740 10 client.5433 fetch_fsmap finished waiting for FSMap version 27 2024-09-17T04:34:29.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.781+0000 7ff52e838740 10 client.5433 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T04:34:29.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.781+0000 7ff52e838740 20 client.5433 populate_metadata read hostname 'smithi037' 2024-09-17T04:34:29.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.781+0000 7ff52e838740 1 --2- 172.21.15.37:0/3784590401 >> [v2:172.21.15.37:6834/4051839052,v1:172.21.15.37:6835/4051839052] conn(0x559ed6201e40 0x559ed65f2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:29.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.781+0000 7ff52e838740 4 client.5433 mds_command: new command op to 5208 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T04:34:29.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.781+0000 7ff52e838740 1 -- 172.21.15.37:0/3784590401 --> [v2:172.21.15.37:6834/4051839052,v1:172.21.15.37:6835/4051839052] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ed655f770 con 0x559ed6201e40 2024-09-17T04:34:29.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.781+0000 7ff524e67640 1 --2- 172.21.15.37:0/3784590401 >> [v2:172.21.15.37:6834/4051839052,v1:172.21.15.37:6835/4051839052] conn(0x559ed6201e40 0x559ed65f2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:29.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.781+0000 7ff4fffff640 10 client.5433 ms_handle_connect on v2:172.21.15.37:6834/4051839052 2024-09-17T04:34:29.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.781+0000 7ff524e67640 1 --2- 172.21.15.37:0/3784590401 >> [v2:172.21.15.37:6834/4051839052,v1:172.21.15.37:6835/4051839052] conn(0x559ed6201e40 0x559ed65f2ec0 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe28fbd3f0ba1e1f server_cookie=bf28c8cca08dc388 in_seq=0 out_seq=0 2024-09-17T04:34:29.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.782+0000 7ff4fffff640 1 -- 172.21.15.37:0/3784590401 <== mds.0 v2:172.21.15.37:6834/4051839052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559ed655f770 con 0x559ed6201e40 2024-09-17T04:34:29.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.782+0000 7ff4fffff640 10 client.5433 handle_command_reply: tid=0 2024-09-17T04:34:29.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.782+0000 7ff52e838740 1 client.5433 shutdown 2024-09-17T04:34:29.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.782+0000 7ff52e838740 1 -- 172.21.15.37:0/3784590401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed621a3b0 msgr2=0x559ed66435b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:29.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.782+0000 7ff52e838740 1 --2- 172.21.15.37:0/3784590401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed621a3b0 0x559ed66435b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff51c012440 tx=0x7ff5200d5280 comp rx=0 tx=0).stop 2024-09-17T04:34:29.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.782+0000 7ff52e838740 1 -- 172.21.15.37:0/3784590401 shutdown_connections 2024-09-17T04:34:29.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.782+0000 7ff52e838740 1 --2- 172.21.15.37:0/3784590401 >> [v2:172.21.15.37:6834/4051839052,v1:172.21.15.37:6835/4051839052] conn(0x559ed6201e40 0x559ed65f2ec0 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.782+0000 7ff52e838740 1 --2- 172.21.15.37:0/3784590401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ed6243270 0x559ed65fe290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.782+0000 7ff52e838740 1 --2- 172.21.15.37:0/3784590401 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559ed6222590 0x559ed663ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.782+0000 7ff4fffff640 0 client.5433 ms_handle_reset on v2:172.21.15.37:6834/4051839052 2024-09-17T04:34:29.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.782+0000 7ff52e838740 1 --2- 172.21.15.37:0/3784590401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed621a3b0 0x559ed66435b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.782+0000 7ff52e838740 1 -- 172.21.15.37:0/3784590401 >> 172.21.15.37:0/3784590401 conn(0x559ed62373c0 msgr2=0x559ed6604fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:29.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.783+0000 7ff52e838740 1 -- 172.21.15.37:0/3784590401 shutdown_connections 2024-09-17T04:34:29.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.783+0000 7ff52e838740 1 -- 172.21.15.37:0/3784590401 wait complete. 2024-09-17T04:34:29.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.783+0000 7ff52e838740 20 client.5433 trim_cache size 0 max 16384 2024-09-17T04:34:29.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.801+0000 7ff52e838740 1 Processor -- start 2024-09-17T04:34:29.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.801+0000 7ff52e838740 1 -- start start 2024-09-17T04:34:29.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.802+0000 7ff52e838740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ed621a3b0 0x559ed6603200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:29.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.802+0000 7ff52e838740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559ed6222590 0x559ed6631fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:29.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.802+0000 7ff52e838740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed6243270 0x559ed6632510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:29.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.802+0000 7ff52e838740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff520202310 con 0x559ed6243270 2024-09-17T04:34:29.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.802+0000 7ff52e838740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff5202015f0 con 0x559ed621a3b0 2024-09-17T04:34:29.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.802+0000 7ff52e838740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7ff520201c40 con 0x559ed6222590 2024-09-17T04:34:29.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.803+0000 7ff525e69640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed6243270 0x559ed6632510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:29.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.803+0000 7ff525e69640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed6243270 0x559ed6632510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48204/0 (socket says 172.21.15.37:48204) 2024-09-17T04:34:29.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.803+0000 7ff525e69640 1 -- 172.21.15.37:0/1135313552 learned_addr learned my addr 172.21.15.37:0/1135313552 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:29.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.803+0000 7ff525e69640 1 -- 172.21.15.37:0/1135313552 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559ed6222590 msgr2=0x559ed6631fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:29.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.803+0000 7ff525e69640 1 --2- 172.21.15.37:0/1135313552 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559ed6222590 0x559ed6631fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.803+0000 7ff525e69640 1 -- 172.21.15.37:0/1135313552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ed621a3b0 msgr2=0x559ed6603200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:29.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.803+0000 7ff525e69640 1 --2- 172.21.15.37:0/1135313552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ed621a3b0 0x559ed6603200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.803+0000 7ff525e69640 1 -- 172.21.15.37:0/1135313552 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ed66430e0 con 0x559ed6243270 2024-09-17T04:34:29.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.803+0000 7ff525e69640 1 --2- 172.21.15.37:0/1135313552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed6243270 0x559ed6632510 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff518009f90 tx=0x7ff51802bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8312bf897b710c3d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:29.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.803+0000 7ff50a7fc640 1 -- 172.21.15.37:0/1135313552 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518043070 con 0x559ed6243270 2024-09-17T04:34:29.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.803+0000 7ff50a7fc640 1 -- 172.21.15.37:0/1135313552 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff518003970 con 0x559ed6243270 2024-09-17T04:34:29.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.803+0000 7ff50a7fc640 1 -- 172.21.15.37:0/1135313552 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518038040 con 0x559ed6243270 2024-09-17T04:34:29.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.804+0000 7ff52e838740 1 -- 172.21.15.37:0/1135313552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed6243270 msgr2=0x559ed6632510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:29.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.804+0000 7ff52e838740 1 --2- 172.21.15.37:0/1135313552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed6243270 0x559ed6632510 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff518009f90 tx=0x7ff51802bf90 comp rx=0 tx=0).stop 2024-09-17T04:34:29.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.804+0000 7ff52e838740 1 -- 172.21.15.37:0/1135313552 shutdown_connections 2024-09-17T04:34:29.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.804+0000 7ff52e838740 1 --2- 172.21.15.37:0/1135313552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed6243270 0x559ed6632510 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.804+0000 7ff52e838740 1 --2- 172.21.15.37:0/1135313552 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559ed6222590 0x559ed6631fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.804+0000 7ff52e838740 1 --2- 172.21.15.37:0/1135313552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ed621a3b0 0x559ed6603200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.804+0000 7ff52e838740 1 -- 172.21.15.37:0/1135313552 >> 172.21.15.37:0/1135313552 conn(0x559ed62373c0 msgr2=0x559ed65eb0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:29.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.804+0000 7ff52e838740 1 -- 172.21.15.37:0/1135313552 shutdown_connections 2024-09-17T04:34:29.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.804+0000 7ff52e838740 1 -- 172.21.15.37:0/1135313552 wait complete. 2024-09-17T04:34:29.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.805+0000 7ff52e838740 1 -- start start 2024-09-17T04:34:29.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.805+0000 7ff52e838740 1 Processor -- start 2024-09-17T04:34:29.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.806+0000 7ff52e838740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed621a3b0 0x559ed66a28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:29.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.806+0000 7ff52e838740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559ed6222590 0x559ed66541a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:29.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.806+0000 7ff52e838740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ed6243270 0x559ed66a8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:29.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.806+0000 7ff52e838740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559ed65fe8c0 con 0x559ed621a3b0 2024-09-17T04:34:29.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.806+0000 7ff52e838740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559ed65fed20 con 0x559ed6243270 2024-09-17T04:34:29.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.806+0000 7ff52e838740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x559ed6629320 con 0x559ed6222590 2024-09-17T04:34:29.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.806+0000 7ff52e838740 10 client.0 _ll_register_callbacks cb 0x7ff51c0016d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:34:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.806+0000 7ff525668640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed621a3b0 0x559ed66a28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.807+0000 7ff525668640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed621a3b0 0x559ed66a28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48216/0 (socket says 172.21.15.37:48216) 2024-09-17T04:34:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.807+0000 7ff525668640 1 -- 172.21.15.37:0/1493133330 learned_addr learned my addr 172.21.15.37:0/1493133330 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.807+0000 7ff524e67640 1 --2- 172.21.15.37:0/1493133330 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559ed6222590 0x559ed66541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.807+0000 7ff525e69640 1 --2- 172.21.15.37:0/1493133330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ed6243270 0x559ed66a8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.807+0000 7ff525668640 1 -- 172.21.15.37:0/1493133330 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559ed6222590 msgr2=0x559ed66541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.807+0000 7ff525668640 1 --2- 172.21.15.37:0/1493133330 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559ed6222590 0x559ed66541a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.807+0000 7ff525668640 1 -- 172.21.15.37:0/1493133330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ed6243270 msgr2=0x559ed66a8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.807+0000 7ff525668640 1 --2- 172.21.15.37:0/1493133330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ed6243270 0x559ed66a8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.807+0000 7ff525668640 1 -- 172.21.15.37:0/1493133330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ed6633af0 con 0x559ed621a3b0 2024-09-17T04:34:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.807+0000 7ff525e69640 1 --2- 172.21.15.37:0/1493133330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ed6243270 0x559ed66a8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.807+0000 7ff525668640 1 --2- 172.21.15.37:0/1493133330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed621a3b0 0x559ed66a28f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff51c018b60 tx=0x7ff51c018b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57567e5601a7e70b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.807+0000 7ff4fffff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-17T04:34:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.807+0000 7ff4fffff640 1 -- 172.21.15.37:0/1493133330 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c018ea0 con 0x559ed621a3b0 2024-09-17T04:34:29.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.807+0000 7ff4fffff640 1 -- 172.21.15.37:0/1493133330 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7ff51c01f820 con 0x559ed621a3b0 2024-09-17T04:34:29.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.807+0000 7ff52e838740 1 -- 172.21.15.37:0/1493133330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff518003970 con 0x559ed621a3b0 2024-09-17T04:34:29.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.807+0000 7ff4fffff640 1 -- 172.21.15.37:0/1493133330 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff51c042050 con 0x559ed621a3b0 2024-09-17T04:34:29.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.807+0000 7ff52e838740 10 client.5439 fetch_fsmap learned FSMap version 27 2024-09-17T04:34:29.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.807+0000 7ff52e838740 1 -- 172.21.15.37:0/1493133330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x559ed65fd990 con 0x559ed621a3b0 2024-09-17T04:34:29.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.808+0000 7ff4fffff640 1 -- 172.21.15.37:0/1493133330 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff518003970 con 0x559ed621a3b0 2024-09-17T04:34:29.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.808+0000 7ff4fffff640 1 -- 172.21.15.37:0/1493133330 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 4147+0+0 (secure 0 0 0) 0x7ff51c045020 con 0x559ed621a3b0 2024-09-17T04:34:29.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.808+0000 7ff52e838740 10 client.5439 fetch_fsmap finished waiting for FSMap version 27 2024-09-17T04:34:29.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.808+0000 7ff52e838740 10 client.5439 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T04:34:29.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.808+0000 7ff52e838740 20 client.5439 populate_metadata read hostname 'smithi037' 2024-09-17T04:34:29.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.808+0000 7ff52e838740 1 --2- 172.21.15.37:0/1493133330 >> [v2:172.21.15.37:6834/4051839052,v1:172.21.15.37:6835/4051839052] conn(0x559ed66a68e0 0x559ed6652ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:29.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.808+0000 7ff52e838740 4 client.5439 mds_command: new command op to 5208 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T04:34:29.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.808+0000 7ff52e838740 1 -- 172.21.15.37:0/1493133330 --> [v2:172.21.15.37:6834/4051839052,v1:172.21.15.37:6835/4051839052] -- command(tid 0: {"prefix": "status"}) -- 0x559ed655b220 con 0x559ed66a68e0 2024-09-17T04:34:29.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.808+0000 7ff524e67640 1 --2- 172.21.15.37:0/1493133330 >> [v2:172.21.15.37:6834/4051839052,v1:172.21.15.37:6835/4051839052] conn(0x559ed66a68e0 0x559ed6652ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:29.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.808+0000 7ff4fffff640 10 client.5439 ms_handle_connect on v2:172.21.15.37:6834/4051839052 2024-09-17T04:34:29.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.808+0000 7ff524e67640 1 --2- 172.21.15.37:0/1493133330 >> [v2:172.21.15.37:6834/4051839052,v1:172.21.15.37:6835/4051839052] conn(0x559ed66a68e0 0x559ed6652ef0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5279900d8bdbe799 server_cookie=44e094ab54ffc7ad in_seq=0 out_seq=0 2024-09-17T04:34:29.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.808+0000 7ff4fffff640 1 -- 172.21.15.37:0/1493133330 <== mds.0 v2:172.21.15.37:6834/4051839052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x559ed655b220 con 0x559ed66a68e0 2024-09-17T04:34:29.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.809+0000 7ff4fffff640 10 client.5439 handle_command_reply: tid=0 2024-09-17T04:34:29.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.809+0000 7ff52e838740 1 client.5439 shutdown 2024-09-17T04:34:29.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.809+0000 7ff52e838740 1 -- 172.21.15.37:0/1493133330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed621a3b0 msgr2=0x559ed66a28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:29.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.809+0000 7ff52e838740 1 --2- 172.21.15.37:0/1493133330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed621a3b0 0x559ed66a28f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff51c018b60 tx=0x7ff51c018b90 comp rx=0 tx=0).stop 2024-09-17T04:34:29.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.809+0000 7ff52e838740 1 -- 172.21.15.37:0/1493133330 shutdown_connections 2024-09-17T04:34:29.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.809+0000 7ff52e838740 1 --2- 172.21.15.37:0/1493133330 >> [v2:172.21.15.37:6834/4051839052,v1:172.21.15.37:6835/4051839052] conn(0x559ed66a68e0 0x559ed6652ef0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.809+0000 7ff52e838740 1 --2- 172.21.15.37:0/1493133330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ed6243270 0x559ed66a8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.809+0000 7ff52e838740 1 --2- 172.21.15.37:0/1493133330 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x559ed6222590 0x559ed66541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.809+0000 7ff4fffff640 0 client.5439 ms_handle_reset on v2:172.21.15.37:6834/4051839052 2024-09-17T04:34:29.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.809+0000 7ff52e838740 1 --2- 172.21.15.37:0/1493133330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559ed621a3b0 0x559ed66a28f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.809+0000 7ff52e838740 1 -- 172.21.15.37:0/1493133330 >> 172.21.15.37:0/1493133330 conn(0x559ed62373c0 msgr2=0x559ed6603200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:29.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.809+0000 7ff52e838740 1 -- 172.21.15.37:0/1493133330 shutdown_connections 2024-09-17T04:34:29.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.809+0000 7ff52e838740 1 -- 172.21.15.37:0/1493133330 wait complete. 2024-09-17T04:34:29.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.809+0000 7ff52e838740 20 client.5439 trim_cache size 0 max 16384 2024-09-17T04:34:29.815 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T04:34:29.815 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "a908a603-430c-4209-8294-0c3460b35f42", 2024-09-17T04:34:29.815 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-09-17T04:34:29.815 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 5208, 2024-09-17T04:34:29.815 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-17T04:34:29.815 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-17T04:34:29.816 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "recovery_fs", 2024-09-17T04:34:29.816 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 4.572866748, 2024-09-17T04:34:29.816 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 27, 2024-09-17T04:34:29.816 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 41, 2024-09-17T04:34:29.816 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 41, 2024-09-17T04:34:29.816 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 13.722933988999999 2024-09-17T04:34:29.816 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T04:34:29.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.809+0000 7ff5278cc640 1 -- 172.21.15.37:0/3775407912 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff504078b90 msgr2=0x7ff50407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:29.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.809+0000 7ff5278cc640 1 --2- 172.21.15.37:0/3775407912 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff504078b90 0x7ff50407b0a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff51c001c20 tx=0x7ff51c001bb0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.809+0000 7ff5278cc640 1 -- 172.21.15.37:0/3775407912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff52014e250 msgr2=0x7ff5201f8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:29.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.809+0000 7ff5278cc640 1 --2- 172.21.15.37:0/3775407912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff52014e250 0x7ff5201f8760 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff510011ae0 tx=0x7ff510009ec0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.810+0000 7ff5278cc640 1 -- 172.21.15.37:0/3775407912 shutdown_connections 2024-09-17T04:34:29.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.810+0000 7ff5278cc640 1 --2- 172.21.15.37:0/3775407912 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7ff504078b90 0x7ff50407b0a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.810+0000 7ff5278cc640 1 --2- 172.21.15.37:0/3775407912 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff5201f8ca0 0x7ff5201f90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.810+0000 7ff5278cc640 1 --2- 172.21.15.37:0/3775407912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff52014e250 0x7ff5201f8760 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.810+0000 7ff5278cc640 1 --2- 172.21.15.37:0/3775407912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff50c004d40 0x7ff520159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.810+0000 7ff5278cc640 1 -- 172.21.15.37:0/3775407912 >> 172.21.15.37:0/3775407912 conn(0x7ff52008bef0 msgr2=0x7ff52015c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:29.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.810+0000 7ff5278cc640 1 -- 172.21.15.37:0/3775407912 shutdown_connections 2024-09-17T04:34:29.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.810+0000 7ff5278cc640 1 -- 172.21.15.37:0/3775407912 wait complete. 2024-09-17T04:34:29.821 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds debug_mds 20 2024-09-17T04:34:29.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.940+0000 7fb18a512640 1 -- 172.21.15.37:0/2220089450 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb184162dc0 msgr2=0x7fb18406dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:29.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.940+0000 7fb18a512640 1 --2- 172.21.15.37:0/2220089450 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb184162dc0 0x7fb18406dea0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb17c009b60 tx=0x7fb17c02a750 comp rx=0 tx=0).stop 2024-09-17T04:34:29.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.941+0000 7fb18a512640 1 -- 172.21.15.37:0/2220089450 shutdown_connections 2024-09-17T04:34:29.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.941+0000 7fb18a512640 1 --2- 172.21.15.37:0/2220089450 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb184162dc0 0x7fb18406dea0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.941+0000 7fb18a512640 1 -- 172.21.15.37:0/2220089450 >> 172.21.15.37:0/2220089450 conn(0x7fb18408bef0 msgr2=0x7fb1840d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:29.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.941+0000 7fb18a512640 1 -- 172.21.15.37:0/2220089450 shutdown_connections 2024-09-17T04:34:29.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.941+0000 7fb18a512640 1 -- 172.21.15.37:0/2220089450 wait complete. 2024-09-17T04:34:29.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.942+0000 7fb18a512640 1 Processor -- start 2024-09-17T04:34:29.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.943+0000 7fb18a512640 1 -- start start 2024-09-17T04:34:29.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.944+0000 7fb18a512640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1840b3530 0x7fb1840b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:29.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.944+0000 7fb18a512640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb1840b9c40 0x7fb18415f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:29.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.944+0000 7fb18a512640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1840b22a0 0x7fb1840b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:29.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.944+0000 7fb18a512640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb1840d33e0 con 0x7fb1840b3530 2024-09-17T04:34:29.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.944+0000 7fb18a512640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb1840d3260 con 0x7fb1840b22a0 2024-09-17T04:34:29.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.944+0000 7fb18a512640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fb1840d3560 con 0x7fb1840b9c40 2024-09-17T04:34:29.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.944+0000 7fb183fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1840b3530 0x7fb1840b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:29.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.944+0000 7fb183fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1840b3530 0x7fb1840b6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48232/0 (socket says 172.21.15.37:48232) 2024-09-17T04:34:29.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.944+0000 7fb183fff640 1 -- 172.21.15.37:0/1412686613 learned_addr learned my addr 172.21.15.37:0/1412686613 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:29.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.944+0000 7fb188aaf640 1 --2- 172.21.15.37:0/1412686613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1840b22a0 0x7fb1840b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:29.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.944+0000 7fb1837fe640 1 --2- 172.21.15.37:0/1412686613 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb1840b9c40 0x7fb18415f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:29.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.944+0000 7fb183fff640 1 -- 172.21.15.37:0/1412686613 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb1840b9c40 msgr2=0x7fb18415f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:29.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.944+0000 7fb183fff640 1 --2- 172.21.15.37:0/1412686613 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb1840b9c40 0x7fb18415f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.944+0000 7fb183fff640 1 -- 172.21.15.37:0/1412686613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1840b22a0 msgr2=0x7fb1840b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:29.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.944+0000 7fb183fff640 1 --2- 172.21.15.37:0/1412686613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1840b22a0 0x7fb1840b26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:29.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.944+0000 7fb183fff640 1 -- 172.21.15.37:0/1412686613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1840b2bf0 con 0x7fb1840b3530 2024-09-17T04:34:29.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.945+0000 7fb183fff640 1 --2- 172.21.15.37:0/1412686613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1840b3530 0x7fb1840b6af0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb17400c500 tx=0x7fb17400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:29.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.945+0000 7fb180ff9640 1 -- 172.21.15.37:0/1412686613 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb174017070 con 0x7fb1840b3530 2024-09-17T04:34:29.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.945+0000 7fb180ff9640 1 -- 172.21.15.37:0/1412686613 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1740035f0 con 0x7fb1840b3530 2024-09-17T04:34:29.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.945+0000 7fb18a512640 1 -- 172.21.15.37:0/1412686613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb18409e970 con 0x7fb1840b3530 2024-09-17T04:34:29.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.945+0000 7fb180ff9640 1 -- 172.21.15.37:0/1412686613 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb174003a60 con 0x7fb1840b3530 2024-09-17T04:34:29.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.945+0000 7fb18a512640 1 -- 172.21.15.37:0/1412686613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1840a0640 con 0x7fb1840b3530 2024-09-17T04:34:29.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.946+0000 7fb18a512640 1 -- 172.21.15.37:0/1412686613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb18406dea0 con 0x7fb1840b3530 2024-09-17T04:34:29.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.947+0000 7fb180ff9640 1 -- 172.21.15.37:0/1412686613 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb17401a070 con 0x7fb1840b3530 2024-09-17T04:34:29.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.947+0000 7fb180ff9640 1 --2- 172.21.15.37:0/1412686613 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb164078b90 0x7fb16407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:29.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.947+0000 7fb180ff9640 1 -- 172.21.15.37:0/1412686613 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7fb17409a2f0 con 0x7fb1840b3530 2024-09-17T04:34:29.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.947+0000 7fb1837fe640 1 --2- 172.21.15.37:0/1412686613 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb164078b90 0x7fb16407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:29.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.948+0000 7fb1837fe640 1 --2- 172.21.15.37:0/1412686613 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb164078b90 0x7fb16407b0a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb1840b6230 tx=0x7fb16c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:29.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:29.952+0000 7fb180ff9640 1 -- 172.21.15.37:0/1412686613 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb1840b2bf0 con 0x7fb1840b3530 2024-09-17T04:34:30.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.139+0000 7fb18a512640 1 -- 172.21.15.37:0/1412686613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7fb18406a150 con 0x7fb1840b3530 2024-09-17T04:34:30.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.140+0000 7fb180ff9640 1 -- 172.21.15.37:0/1412686613 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v32)=0 v32) ==== 108+0+0 (secure 0 0 0) 0x7fb174061fe0 con 0x7fb1840b3530 2024-09-17T04:34:30.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.142+0000 7fb18a512640 1 -- 172.21.15.37:0/1412686613 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb164078b90 msgr2=0x7fb16407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:30.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.142+0000 7fb18a512640 1 --2- 172.21.15.37:0/1412686613 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb164078b90 0x7fb16407b0a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb1840b6230 tx=0x7fb16c009040 comp rx=0 tx=0).stop 2024-09-17T04:34:30.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.142+0000 7fb18a512640 1 -- 172.21.15.37:0/1412686613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1840b3530 msgr2=0x7fb1840b6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:30.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.142+0000 7fb18a512640 1 --2- 172.21.15.37:0/1412686613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1840b3530 0x7fb1840b6af0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb17400c500 tx=0x7fb17400c9c0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.142+0000 7fb18a512640 1 -- 172.21.15.37:0/1412686613 shutdown_connections 2024-09-17T04:34:30.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.142+0000 7fb18a512640 1 --2- 172.21.15.37:0/1412686613 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb164078b90 0x7fb16407b0a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.142+0000 7fb18a512640 1 --2- 172.21.15.37:0/1412686613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1840b22a0 0x7fb1840b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.142+0000 7fb18a512640 1 --2- 172.21.15.37:0/1412686613 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb1840b9c40 0x7fb18415f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.142+0000 7fb18a512640 1 --2- 172.21.15.37:0/1412686613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1840b3530 0x7fb1840b6af0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.142+0000 7fb18a512640 1 -- 172.21.15.37:0/1412686613 >> 172.21.15.37:0/1412686613 conn(0x7fb18408bef0 msgr2=0x7fb1840ca170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:30.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.142+0000 7fb18a512640 1 -- 172.21.15.37:0/1412686613 shutdown_connections 2024-09-17T04:34:30.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.142+0000 7fb18a512640 1 -- 172.21.15.37:0/1412686613 wait complete. 2024-09-17T04:34:30.153 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / force,recursive,repair 2024-09-17T04:34:30.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.274+0000 7fa7158e6640 1 -- 172.21.15.37:0/650385762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7100b6750 msgr2=0x7fa710161ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:30.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.274+0000 7fa7158e6640 1 --2- 172.21.15.37:0/650385762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7100b6750 0x7fa710161ff0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa704009820 tx=0x7fa70402a540 comp rx=0 tx=0).stop 2024-09-17T04:34:30.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.274+0000 7fa7158e6640 1 -- 172.21.15.37:0/650385762 shutdown_connections 2024-09-17T04:34:30.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.274+0000 7fa7158e6640 1 --2- 172.21.15.37:0/650385762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7100b6750 0x7fa710161ff0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.274+0000 7fa7158e6640 1 -- 172.21.15.37:0/650385762 >> 172.21.15.37:0/650385762 conn(0x7fa71008bef0 msgr2=0x7fa7100d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:30.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.274+0000 7fa7158e6640 1 -- 172.21.15.37:0/650385762 shutdown_connections 2024-09-17T04:34:30.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.274+0000 7fa7158e6640 1 -- 172.21.15.37:0/650385762 wait complete. 2024-09-17T04:34:30.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.276+0000 7fa7158e6640 1 Processor -- start 2024-09-17T04:34:30.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.276+0000 7fa7158e6640 1 -- start start 2024-09-17T04:34:30.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.277+0000 7fa7158e6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7100b3a10 0x7fa7101f9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:30.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.277+0000 7fa7158e6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa71015edc0 0x7fa7100ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:30.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.277+0000 7fa7158e6640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa7100c84e0 0x7fa7101f5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:30.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.277+0000 7fa7158e6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa7100d3020 con 0x7fa71015edc0 2024-09-17T04:34:30.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.277+0000 7fa7158e6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa7100d2ea0 con 0x7fa7100b3a10 2024-09-17T04:34:30.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.277+0000 7fa7158e6640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fa7100d31a0 con 0x7fa7100c84e0 2024-09-17T04:34:30.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.278+0000 7fa70e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa71015edc0 0x7fa7100ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:30.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.278+0000 7fa70e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa71015edc0 0x7fa7100ce130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48248/0 (socket says 172.21.15.37:48248) 2024-09-17T04:34:30.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.278+0000 7fa70e7fc640 1 -- 172.21.15.37:0/4204418530 learned_addr learned my addr 172.21.15.37:0/4204418530 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:30.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.278+0000 7fa70effd640 1 --2- 172.21.15.37:0/4204418530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7100b3a10 0x7fa7101f9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:30.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.278+0000 7fa70f7fe640 1 --2- 172.21.15.37:0/4204418530 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa7100c84e0 0x7fa7101f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:30.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.278+0000 7fa70e7fc640 1 -- 172.21.15.37:0/4204418530 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa7100c84e0 msgr2=0x7fa7101f5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:30.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.278+0000 7fa70e7fc640 1 --2- 172.21.15.37:0/4204418530 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa7100c84e0 0x7fa7101f5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.278+0000 7fa70e7fc640 1 -- 172.21.15.37:0/4204418530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7100b3a10 msgr2=0x7fa7101f9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:30.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.278+0000 7fa70e7fc640 1 --2- 172.21.15.37:0/4204418530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7100b3a10 0x7fa7101f9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.278+0000 7fa70e7fc640 1 -- 172.21.15.37:0/4204418530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7100c9ef0 con 0x7fa71015edc0 2024-09-17T04:34:30.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.278+0000 7fa70effd640 1 --2- 172.21.15.37:0/4204418530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7100b3a10 0x7fa7101f9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:30.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.278+0000 7fa70e7fc640 1 --2- 172.21.15.37:0/4204418530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa71015edc0 0x7fa7100ce130 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa704009910 tx=0x7fa704003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:30.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.278+0000 7fa6f7fff640 1 -- 172.21.15.37:0/4204418530 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70403f070 con 0x7fa71015edc0 2024-09-17T04:34:30.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.278+0000 7fa6f7fff640 1 -- 172.21.15.37:0/4204418530 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa704004ca0 con 0x7fa71015edc0 2024-09-17T04:34:30.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.278+0000 7fa7158e6640 1 -- 172.21.15.37:0/4204418530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7101f8a50 con 0x7fa71015edc0 2024-09-17T04:34:30.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.278+0000 7fa6f7fff640 1 -- 172.21.15.37:0/4204418530 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa704005c30 con 0x7fa71015edc0 2024-09-17T04:34:30.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.279+0000 7fa7158e6640 1 -- 172.21.15.37:0/4204418530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7100ce670 con 0x7fa71015edc0 2024-09-17T04:34:30.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.280+0000 7fa6f7fff640 1 -- 172.21.15.37:0/4204418530 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa704042070 con 0x7fa71015edc0 2024-09-17T04:34:30.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.281+0000 7fa6f7fff640 1 --2- 172.21.15.37:0/4204418530 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa6fc078b90 0x7fa6fc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:30.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.281+0000 7fa6f7fff640 1 -- 172.21.15.37:0/4204418530 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7fa7040bbed0 con 0x7fa71015edc0 2024-09-17T04:34:30.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.281+0000 7fa70effd640 1 --2- 172.21.15.37:0/4204418530 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa6fc078b90 0x7fa6fc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:30.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.282+0000 7fa70effd640 1 --2- 172.21.15.37:0/4204418530 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa6fc078b90 0x7fa6fc07b0a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa700007fd0 tx=0x7fa700009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:30.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.282+0000 7fa71c854740 1 Processor -- start 2024-09-17T04:34:30.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.282+0000 7fa71c854740 1 -- start start 2024-09-17T04:34:30.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.283+0000 7fa71c854740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9cc4d0 0x55f1dadb99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:30.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.283+0000 7fa71c854740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1da9a3610 0x55f1dad8e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:30.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.283+0000 7fa71c854740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55f1da9ab7f0 0x55f1dad8ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:30.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.283+0000 7fa71c854740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f1dace6a30 con 0x55f1da9cc4d0 2024-09-17T04:34:30.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.283+0000 7fa71c854740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f1dacd5e60 con 0x55f1da9a3610 2024-09-17T04:34:30.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.283+0000 7fa71c854740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x55f1dacd9c70 con 0x55f1da9ab7f0 2024-09-17T04:34:30.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.283+0000 7fa70effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9cc4d0 0x55f1dadb99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:30.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.284+0000 7fa70effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9cc4d0 0x55f1dadb99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48262/0 (socket says 172.21.15.37:48262) 2024-09-17T04:34:30.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.284+0000 7fa70effd640 1 -- 172.21.15.37:0/2606377712 learned_addr learned my addr 172.21.15.37:0/2606377712 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:30.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.284+0000 7fa70f7fe640 1 --2- 172.21.15.37:0/2606377712 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55f1da9ab7f0 0x55f1dad8ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:30.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.284+0000 7fa70e7fc640 1 --2- 172.21.15.37:0/2606377712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1da9a3610 0x55f1dad8e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:30.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.284+0000 7fa70effd640 1 -- 172.21.15.37:0/2606377712 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55f1da9ab7f0 msgr2=0x55f1dad8ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:30.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.284+0000 7fa70effd640 1 --2- 172.21.15.37:0/2606377712 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55f1da9ab7f0 0x55f1dad8ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.284+0000 7fa70effd640 1 -- 172.21.15.37:0/2606377712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1da9a3610 msgr2=0x55f1dad8e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:30.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.284+0000 7fa70effd640 1 --2- 172.21.15.37:0/2606377712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1da9a3610 0x55f1dad8e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.284+0000 7fa70effd640 1 -- 172.21.15.37:0/2606377712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1dac7c190 con 0x55f1da9cc4d0 2024-09-17T04:34:30.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.284+0000 7fa70effd640 1 --2- 172.21.15.37:0/2606377712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9cc4d0 0x55f1dadb99a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa700005100 tx=0x7fa700011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43896bae57d4762b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:30.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.284+0000 7fa6f57fa640 1 -- 172.21.15.37:0/2606377712 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7000035f0 con 0x55f1da9cc4d0 2024-09-17T04:34:30.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.284+0000 7fa6f57fa640 1 -- 172.21.15.37:0/2606377712 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa700003880 con 0x55f1da9cc4d0 2024-09-17T04:34:30.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.284+0000 7fa6f57fa640 1 -- 172.21.15.37:0/2606377712 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700003c30 con 0x55f1da9cc4d0 2024-09-17T04:34:30.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.285+0000 7fa71c854740 1 -- 172.21.15.37:0/2606377712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9cc4d0 msgr2=0x55f1dadb99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:30.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.285+0000 7fa71c854740 1 --2- 172.21.15.37:0/2606377712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9cc4d0 0x55f1dadb99a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa700005100 tx=0x7fa700011000 comp rx=0 tx=0).stop 2024-09-17T04:34:30.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.285+0000 7fa71c854740 1 -- 172.21.15.37:0/2606377712 shutdown_connections 2024-09-17T04:34:30.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.285+0000 7fa71c854740 1 --2- 172.21.15.37:0/2606377712 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55f1da9ab7f0 0x55f1dad8ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.285+0000 7fa71c854740 1 --2- 172.21.15.37:0/2606377712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1da9a3610 0x55f1dad8e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.285+0000 7fa71c854740 1 --2- 172.21.15.37:0/2606377712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9cc4d0 0x55f1dadb99a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.285+0000 7fa71c854740 1 -- 172.21.15.37:0/2606377712 >> 172.21.15.37:0/2606377712 conn(0x55f1da9c0620 msgr2=0x55f1dad74c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:30.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.285+0000 7fa71c854740 1 -- 172.21.15.37:0/2606377712 shutdown_connections 2024-09-17T04:34:30.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.285+0000 7fa71c854740 1 -- 172.21.15.37:0/2606377712 wait complete. 2024-09-17T04:34:30.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.286+0000 7fa71c854740 1 -- start start 2024-09-17T04:34:30.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.286+0000 7fa71c854740 1 Processor -- start 2024-09-17T04:34:30.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.287+0000 7fa71c854740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55f1da9a3610 0x55f1dadaf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:30.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.287+0000 7fa71c854740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9ab7f0 0x55f1dadb5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:30.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.287+0000 7fa71c854740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1da9cc4d0 0x55f1dad9eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:30.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.288+0000 7fa71c854740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f1daa46880 con 0x55f1da9ab7f0 2024-09-17T04:34:30.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.288+0000 7fa71c854740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f1dac5c5a0 con 0x55f1da9cc4d0 2024-09-17T04:34:30.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.288+0000 7fa71c854740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x55f1dad9c210 con 0x55f1da9a3610 2024-09-17T04:34:30.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.288+0000 7fa71c854740 10 client.0 _ll_register_callbacks cb 0x55f1dac8aa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:34:30.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.288+0000 7fa70e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9ab7f0 0x55f1dadb5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:30.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.288+0000 7fa70e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9ab7f0 0x55f1dadb5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48274/0 (socket says 172.21.15.37:48274) 2024-09-17T04:34:30.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.288+0000 7fa70e7fc640 1 -- 172.21.15.37:0/1129694460 learned_addr learned my addr 172.21.15.37:0/1129694460 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:30.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.288+0000 7fa70effd640 1 --2- 172.21.15.37:0/1129694460 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55f1da9a3610 0x55f1dadaf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:30.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.288+0000 7fa70f7fe640 1 --2- 172.21.15.37:0/1129694460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1da9cc4d0 0x55f1dad9eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:30.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.288+0000 7fa70e7fc640 1 -- 172.21.15.37:0/1129694460 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55f1da9a3610 msgr2=0x55f1dadaf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:30.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.288+0000 7fa70e7fc640 1 --2- 172.21.15.37:0/1129694460 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55f1da9a3610 0x55f1dadaf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.288+0000 7fa70e7fc640 1 -- 172.21.15.37:0/1129694460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1da9cc4d0 msgr2=0x55f1dad9eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:30.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.288+0000 7fa70e7fc640 1 --2- 172.21.15.37:0/1129694460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1da9cc4d0 0x55f1dad9eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.288+0000 7fa70e7fc640 1 -- 172.21.15.37:0/1129694460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1daab0940 con 0x55f1da9ab7f0 2024-09-17T04:34:30.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.288+0000 7fa70e7fc640 1 --2- 172.21.15.37:0/1129694460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9ab7f0 0x55f1dadb5940 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa7040ba1c0 tx=0x7fa704083f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d6a5a834d44144 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:30.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.288+0000 7fa6ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-17T04:34:30.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.288+0000 7fa6ee7fc640 1 -- 172.21.15.37:0/1129694460 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa704049050 con 0x55f1da9ab7f0 2024-09-17T04:34:30.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.288+0000 7fa6ee7fc640 1 -- 172.21.15.37:0/1129694460 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7fa70408cf60 con 0x55f1da9ab7f0 2024-09-17T04:34:30.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.288+0000 7fa71c854740 1 -- 172.21.15.37:0/1129694460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f1dacac420 con 0x55f1da9ab7f0 2024-09-17T04:34:30.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.288+0000 7fa6ee7fc640 1 -- 172.21.15.37:0/1129694460 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa704004e90 con 0x55f1da9ab7f0 2024-09-17T04:34:30.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.288+0000 7fa71c854740 10 client.5457 fetch_fsmap learned FSMap version 27 2024-09-17T04:34:30.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.288+0000 7fa71c854740 1 -- 172.21.15.37:0/1129694460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x55f1dab110d0 con 0x55f1da9ab7f0 2024-09-17T04:34:30.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.289+0000 7fa6ee7fc640 1 -- 172.21.15.37:0/1129694460 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55f1dacac420 con 0x55f1da9ab7f0 2024-09-17T04:34:30.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.289+0000 7fa6ee7fc640 1 -- 172.21.15.37:0/1129694460 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 4147+0+0 (secure 0 0 0) 0x7fa704099b80 con 0x55f1da9ab7f0 2024-09-17T04:34:30.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.289+0000 7fa71c854740 10 client.5457 fetch_fsmap finished waiting for FSMap version 27 2024-09-17T04:34:30.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.289+0000 7fa71c854740 10 client.5457 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-09-17T04:34:30.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.289+0000 7fa71c854740 20 client.5457 populate_metadata read hostname 'smithi037' 2024-09-17T04:34:30.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.289+0000 7fa71c854740 1 --2- 172.21.15.37:0/1129694460 >> [v2:172.21.15.37:6834/4051839052,v1:172.21.15.37:6835/4051839052] conn(0x55f1da98b2b0 0x55f1dadcb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:30.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.289+0000 7fa71c854740 4 client.5457 mds_command: new command op to 5208 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T04:34:30.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.289+0000 7fa71c854740 1 -- 172.21.15.37:0/1129694460 --> [v2:172.21.15.37:6834/4051839052,v1:172.21.15.37:6835/4051839052] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1dace8980 con 0x55f1da98b2b0 2024-09-17T04:34:30.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.289+0000 7fa70effd640 1 --2- 172.21.15.37:0/1129694460 >> [v2:172.21.15.37:6834/4051839052,v1:172.21.15.37:6835/4051839052] conn(0x55f1da98b2b0 0x55f1dadcb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:30.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.289+0000 7fa6ee7fc640 10 client.5457 ms_handle_connect on v2:172.21.15.37:6834/4051839052 2024-09-17T04:34:30.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.289+0000 7fa70effd640 1 --2- 172.21.15.37:0/1129694460 >> [v2:172.21.15.37:6834/4051839052,v1:172.21.15.37:6835/4051839052] conn(0x55f1da98b2b0 0x55f1dadcb8c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1df9aeee6eb0fcd7 server_cookie=a1f88838ff8f3ecf in_seq=0 out_seq=0 2024-09-17T04:34:30.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.290+0000 7fa6ee7fc640 1 -- 172.21.15.37:0/1129694460 <== mds.0 v2:172.21.15.37:6834/4051839052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f1dace8980 con 0x55f1da98b2b0 2024-09-17T04:34:30.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.290+0000 7fa6ee7fc640 10 client.5457 handle_command_reply: tid=0 2024-09-17T04:34:30.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.290+0000 7fa71c854740 1 client.5457 shutdown 2024-09-17T04:34:30.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.290+0000 7fa71c854740 1 -- 172.21.15.37:0/1129694460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9ab7f0 msgr2=0x55f1dadb5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:30.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.290+0000 7fa71c854740 1 --2- 172.21.15.37:0/1129694460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9ab7f0 0x55f1dadb5940 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa7040ba1c0 tx=0x7fa704083f70 comp rx=0 tx=0).stop 2024-09-17T04:34:30.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.290+0000 7fa71c854740 1 -- 172.21.15.37:0/1129694460 shutdown_connections 2024-09-17T04:34:30.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.290+0000 7fa71c854740 1 --2- 172.21.15.37:0/1129694460 >> [v2:172.21.15.37:6834/4051839052,v1:172.21.15.37:6835/4051839052] conn(0x55f1da98b2b0 0x55f1dadcb8c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.290+0000 7fa71c854740 1 --2- 172.21.15.37:0/1129694460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1da9cc4d0 0x55f1dad9eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.290+0000 7fa71c854740 1 --2- 172.21.15.37:0/1129694460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9ab7f0 0x55f1dadb5940 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.290+0000 7fa71c854740 1 --2- 172.21.15.37:0/1129694460 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55f1da9a3610 0x55f1dadaf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.290+0000 7fa6ee7fc640 0 client.5457 ms_handle_reset on v2:172.21.15.37:6834/4051839052 2024-09-17T04:34:30.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.290+0000 7fa71c854740 1 -- 172.21.15.37:0/1129694460 >> 172.21.15.37:0/1129694460 conn(0x55f1da9c0620 msgr2=0x55f1dad8e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:30.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.290+0000 7fa71c854740 1 -- 172.21.15.37:0/1129694460 shutdown_connections 2024-09-17T04:34:30.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.290+0000 7fa71c854740 1 -- 172.21.15.37:0/1129694460 wait complete. 2024-09-17T04:34:30.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.290+0000 7fa71c854740 20 client.5457 trim_cache size 0 max 16384 2024-09-17T04:34:30.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.309+0000 7fa71c854740 1 Processor -- start 2024-09-17T04:34:30.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.309+0000 7fa71c854740 1 -- start start 2024-09-17T04:34:30.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.310+0000 7fa71c854740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9c0620 0x55f1dae30be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:30.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.310+0000 7fa71c854740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55f1da9a3610 0x55f1dad6f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:30.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.310+0000 7fa71c854740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1da9ab7f0 0x55f1dad6fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:30.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.310+0000 7fa71c854740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa710203b00 con 0x55f1da9c0620 2024-09-17T04:34:30.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.310+0000 7fa71c854740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa710202220 con 0x55f1da9ab7f0 2024-09-17T04:34:30.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.310+0000 7fa71c854740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fa7102015b0 con 0x55f1da9a3610 2024-09-17T04:34:30.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.311+0000 7fa70effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9c0620 0x55f1dae30be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:30.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.311+0000 7fa70effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9c0620 0x55f1dae30be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48280/0 (socket says 172.21.15.37:48280) 2024-09-17T04:34:30.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.311+0000 7fa70effd640 1 -- 172.21.15.37:0/3596259342 learned_addr learned my addr 172.21.15.37:0/3596259342 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:30.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.311+0000 7fa70e7fc640 1 --2- 172.21.15.37:0/3596259342 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55f1da9a3610 0x55f1dad6f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:30.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.311+0000 7fa70f7fe640 1 --2- 172.21.15.37:0/3596259342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1da9ab7f0 0x55f1dad6fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:30.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.311+0000 7fa70effd640 1 -- 172.21.15.37:0/3596259342 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55f1da9a3610 msgr2=0x55f1dad6f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:30.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.311+0000 7fa70effd640 1 --2- 172.21.15.37:0/3596259342 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55f1da9a3610 0x55f1dad6f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.311+0000 7fa70effd640 1 -- 172.21.15.37:0/3596259342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1da9ab7f0 msgr2=0x55f1dad6fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:30.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.311+0000 7fa70effd640 1 --2- 172.21.15.37:0/3596259342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1da9ab7f0 0x55f1dad6fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.311+0000 7fa70effd640 1 -- 172.21.15.37:0/3596259342 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1dad74de0 con 0x55f1da9c0620 2024-09-17T04:34:30.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.311+0000 7fa70effd640 1 --2- 172.21.15.37:0/3596259342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9c0620 0x55f1dae30be0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa700014000 tx=0x7fa700001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f065cb1b9474986 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:30.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.311+0000 7fa6f4ff9640 1 -- 172.21.15.37:0/3596259342 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700003ba0 con 0x55f1da9c0620 2024-09-17T04:34:30.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.311+0000 7fa6f4ff9640 1 -- 172.21.15.37:0/3596259342 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa700027040 con 0x55f1da9c0620 2024-09-17T04:34:30.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.311+0000 7fa6f4ff9640 1 -- 172.21.15.37:0/3596259342 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700007260 con 0x55f1da9c0620 2024-09-17T04:34:30.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.312+0000 7fa71c854740 1 -- 172.21.15.37:0/3596259342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9c0620 msgr2=0x55f1dae30be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:30.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.312+0000 7fa71c854740 1 --2- 172.21.15.37:0/3596259342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9c0620 0x55f1dae30be0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa700014000 tx=0x7fa700001080 comp rx=0 tx=0).stop 2024-09-17T04:34:30.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.312+0000 7fa71c854740 1 -- 172.21.15.37:0/3596259342 shutdown_connections 2024-09-17T04:34:30.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.312+0000 7fa71c854740 1 --2- 172.21.15.37:0/3596259342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1da9ab7f0 0x55f1dad6fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.312+0000 7fa71c854740 1 --2- 172.21.15.37:0/3596259342 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55f1da9a3610 0x55f1dad6f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.312+0000 7fa71c854740 1 --2- 172.21.15.37:0/3596259342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9c0620 0x55f1dae30be0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.312+0000 7fa71c854740 1 -- 172.21.15.37:0/3596259342 >> 172.21.15.37:0/3596259342 conn(0x55f1dad9bbe0 msgr2=0x55f1dad8cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:30.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.312+0000 7fa71c854740 1 -- 172.21.15.37:0/3596259342 shutdown_connections 2024-09-17T04:34:30.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.312+0000 7fa71c854740 1 -- 172.21.15.37:0/3596259342 wait complete. 2024-09-17T04:34:30.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.313+0000 7fa71c854740 1 -- start start 2024-09-17T04:34:30.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.313+0000 7fa71c854740 1 Processor -- start 2024-09-17T04:34:30.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.314+0000 7fa71c854740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9a3610 0x55f1dad8d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:30.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.314+0000 7fa71c854740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1da9ab7f0 0x55f1dadcf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:30.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.314+0000 7fa71c854740 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55f1da9c0620 0x55f1dae2e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:30.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.314+0000 7fa71c854740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f1dad9ebe0 con 0x55f1da9a3610 2024-09-17T04:34:30.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.314+0000 7fa71c854740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa710201c00 con 0x55f1da9ab7f0 2024-09-17T04:34:30.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.314+0000 7fa71c854740 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x55f1dadcfc50 con 0x55f1da9c0620 2024-09-17T04:34:30.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.314+0000 7fa71c854740 10 client.0 _ll_register_callbacks cb 0x7fa704005eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T04:34:30.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.315+0000 7fa70effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9a3610 0x55f1dad8d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:30.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.315+0000 7fa70effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9a3610 0x55f1dad8d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48286/0 (socket says 172.21.15.37:48286) 2024-09-17T04:34:30.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.315+0000 7fa70effd640 1 -- 172.21.15.37:0/1163919441 learned_addr learned my addr 172.21.15.37:0/1163919441 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:30.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.315+0000 7fa70e7fc640 1 --2- 172.21.15.37:0/1163919441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1da9ab7f0 0x55f1dadcf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:30.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.315+0000 7fa70f7fe640 1 --2- 172.21.15.37:0/1163919441 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55f1da9c0620 0x55f1dae2e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:30.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.315+0000 7fa70effd640 1 -- 172.21.15.37:0/1163919441 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55f1da9c0620 msgr2=0x55f1dae2e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:30.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.315+0000 7fa70effd640 1 --2- 172.21.15.37:0/1163919441 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55f1da9c0620 0x55f1dae2e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.315+0000 7fa70effd640 1 -- 172.21.15.37:0/1163919441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1da9ab7f0 msgr2=0x55f1dadcf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T04:34:30.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.315+0000 7fa70effd640 1 --2- 172.21.15.37:0/1163919441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1da9ab7f0 0x55f1dadcf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.315+0000 7fa70effd640 1 -- 172.21.15.37:0/1163919441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1dad6ae60 con 0x55f1da9a3610 2024-09-17T04:34:30.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.315+0000 7fa70e7fc640 1 --2- 172.21.15.37:0/1163919441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1da9ab7f0 0x55f1dadcf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:30.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.315+0000 7fa70effd640 1 --2- 172.21.15.37:0/1163919441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9a3610 0x55f1dad8d380 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa700005040 tx=0x7fa700003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f02f9c1dcc3a91 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:30.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.315+0000 7fa6ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-17T04:34:30.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.315+0000 7fa6ee7fc640 1 -- 172.21.15.37:0/1163919441 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700007020 con 0x55f1da9a3610 2024-09-17T04:34:30.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.315+0000 7fa6ee7fc640 1 -- 172.21.15.37:0/1163919441 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7fa70003dd50 con 0x55f1da9a3610 2024-09-17T04:34:30.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.315+0000 7fa71c854740 1 -- 172.21.15.37:0/1163919441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa700027040 con 0x55f1da9a3610 2024-09-17T04:34:30.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.315+0000 7fa6ee7fc640 1 -- 172.21.15.37:0/1163919441 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa7000071c0 con 0x55f1da9a3610 2024-09-17T04:34:30.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.315+0000 7fa71c854740 10 client.5463 fetch_fsmap learned FSMap version 27 2024-09-17T04:34:30.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.315+0000 7fa71c854740 1 -- 172.21.15.37:0/1163919441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x55f1daa157e0 con 0x55f1da9a3610 2024-09-17T04:34:30.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.316+0000 7fa6ee7fc640 1 -- 172.21.15.37:0/1163919441 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa700027040 con 0x55f1da9a3610 2024-09-17T04:34:30.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.316+0000 7fa6ee7fc640 1 -- 172.21.15.37:0/1163919441 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 4147+0+0 (secure 0 0 0) 0x7fa700007360 con 0x55f1da9a3610 2024-09-17T04:34:30.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.316+0000 7fa71c854740 10 client.5463 fetch_fsmap finished waiting for FSMap version 27 2024-09-17T04:34:30.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.316+0000 7fa71c854740 10 client.5463 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-09-17T04:34:30.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.316+0000 7fa71c854740 20 client.5463 populate_metadata read hostname 'smithi037' 2024-09-17T04:34:30.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.316+0000 7fa71c854740 1 --2- 172.21.15.37:0/1163919441 >> [v2:172.21.15.37:6834/4051839052,v1:172.21.15.37:6835/4051839052] conn(0x55f1da9606f0 0x55f1daddb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:30.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.316+0000 7fa71c854740 4 client.5463 mds_command: new command op to 5208 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive", "repair"]}] 2024-09-17T04:34:30.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.316+0000 7fa71c854740 1 -- 172.21.15.37:0/1163919441 --> [v2:172.21.15.37:6834/4051839052,v1:172.21.15.37:6835/4051839052] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive", "repair"]}) -- 0x55f1dace4430 con 0x55f1da9606f0 2024-09-17T04:34:30.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.316+0000 7fa70e7fc640 1 --2- 172.21.15.37:0/1163919441 >> [v2:172.21.15.37:6834/4051839052,v1:172.21.15.37:6835/4051839052] conn(0x55f1da9606f0 0x55f1daddb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:30.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.316+0000 7fa6ee7fc640 10 client.5463 ms_handle_connect on v2:172.21.15.37:6834/4051839052 2024-09-17T04:34:30.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.316+0000 7fa70e7fc640 1 --2- 172.21.15.37:0/1163919441 >> [v2:172.21.15.37:6834/4051839052,v1:172.21.15.37:6835/4051839052] conn(0x55f1da9606f0 0x55f1daddb110 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9ba54c5e9408e86 server_cookie=453249ecbc5eb170 in_seq=0 out_seq=0 2024-09-17T04:34:30.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.317+0000 7fa6ee7fc640 1 -- 172.21.15.37:0/1163919441 <== mds.0 v2:172.21.15.37:6834/4051839052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f1dace4430 con 0x55f1da9606f0 2024-09-17T04:34:30.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.317+0000 7fa6ee7fc640 10 client.5463 handle_command_reply: tid=0 2024-09-17T04:34:30.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.317+0000 7fa71c854740 1 client.5463 shutdown 2024-09-17T04:34:30.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.317+0000 7fa71c854740 1 -- 172.21.15.37:0/1163919441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9a3610 msgr2=0x55f1dad8d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:30.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.317+0000 7fa71c854740 1 --2- 172.21.15.37:0/1163919441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9a3610 0x55f1dad8d380 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa700005040 tx=0x7fa700003e30 comp rx=0 tx=0).stop 2024-09-17T04:34:30.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.317+0000 7fa71c854740 1 -- 172.21.15.37:0/1163919441 shutdown_connections 2024-09-17T04:34:30.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.317+0000 7fa71c854740 1 --2- 172.21.15.37:0/1163919441 >> [v2:172.21.15.37:6834/4051839052,v1:172.21.15.37:6835/4051839052] conn(0x55f1da9606f0 0x55f1daddb110 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.317+0000 7fa71c854740 1 --2- 172.21.15.37:0/1163919441 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55f1da9c0620 0x55f1dae2e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.317+0000 7fa71c854740 1 --2- 172.21.15.37:0/1163919441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f1da9ab7f0 0x55f1dadcf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.317+0000 7fa71c854740 1 --2- 172.21.15.37:0/1163919441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f1da9a3610 0x55f1dad8d380 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.317+0000 7fa6ee7fc640 0 client.5463 ms_handle_reset on v2:172.21.15.37:6834/4051839052 2024-09-17T04:34:30.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.317+0000 7fa71c854740 1 -- 172.21.15.37:0/1163919441 >> 172.21.15.37:0/1163919441 conn(0x55f1dad9bbe0 msgr2=0x55f1dae301c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:30.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.318+0000 7fa71c854740 1 -- 172.21.15.37:0/1163919441 shutdown_connections 2024-09-17T04:34:30.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.318+0000 7fa71c854740 1 -- 172.21.15.37:0/1163919441 wait complete. 2024-09-17T04:34:30.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.318+0000 7fa71c854740 20 client.5463 trim_cache size 0 max 16384 2024-09-17T04:34:30.323 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T04:34:30.323 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-09-17T04:34:30.323 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "300dcc70-7093-4745-8d49-e41d3b43d6cc", 2024-09-17T04:34:30.323 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-09-17T04:34:30.323 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T04:34:30.323 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-09-17T04:34:30.318+0000 7f55101d1640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2024-09-17T04:34:30.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.318+0000 7fa7158e6640 1 -- 172.21.15.37:0/4204418530 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa6fc078b90 msgr2=0x7fa6fc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:30.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.318+0000 7fa7158e6640 1 --2- 172.21.15.37:0/4204418530 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa6fc078b90 0x7fa6fc07b0a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa700007fd0 tx=0x7fa700009040 comp rx=0 tx=0).stop 2024-09-17T04:34:30.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.318+0000 7fa7158e6640 1 -- 172.21.15.37:0/4204418530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa71015edc0 msgr2=0x7fa7100ce130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:30.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.318+0000 7fa7158e6640 1 --2- 172.21.15.37:0/4204418530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa71015edc0 0x7fa7100ce130 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa704009910 tx=0x7fa704003c10 comp rx=0 tx=0).stop 2024-09-17T04:34:30.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.318+0000 7fa7158e6640 1 -- 172.21.15.37:0/4204418530 shutdown_connections 2024-09-17T04:34:30.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.318+0000 7fa7158e6640 1 --2- 172.21.15.37:0/4204418530 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa6fc078b90 0x7fa6fc07b0a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.318+0000 7fa7158e6640 1 --2- 172.21.15.37:0/4204418530 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa7100c84e0 0x7fa7101f5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.318+0000 7fa7158e6640 1 --2- 172.21.15.37:0/4204418530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa71015edc0 0x7fa7100ce130 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.318+0000 7fa7158e6640 1 --2- 172.21.15.37:0/4204418530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7100b3a10 0x7fa7101f9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.318+0000 7fa7158e6640 1 -- 172.21.15.37:0/4204418530 >> 172.21.15.37:0/4204418530 conn(0x7fa71008bef0 msgr2=0x7fa7100b9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:30.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.319+0000 7fa7158e6640 1 -- 172.21.15.37:0/4204418530 shutdown_connections 2024-09-17T04:34:30.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.319+0000 7fa7158e6640 1 -- 172.21.15.37:0/4204418530 wait complete. 2024-09-17T04:34:30.325 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-09-17T04:34:30.319+0000 7f55101d1640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55b7e6409180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2024-09-17T04:34:30.326 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-09-17T04:34:30.320+0000 7f55101d1640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v1 ap=1 DIRTYPARENT s=6291456 n() | request=0 lock=0 dirtyparent=1 authpin=1 scrubqueue=0 0x55b7e6690c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(5)0x10000000001:[<0x10000000000/sixmegs v8>,<0x1/subdir v10>]//[]","memoryvalue":"(5)0x10000000001:[<0x10000000000/sixmegs v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-17T04:34:30.326 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-09-17T04:34:30.324+0000 7f55101d1640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v4 pv6 ap=5 snaprealm=0x55b7e66066c0 DIRTYPARENT f(v1 1=0+1) n(v0 1=0+1)/n() (inest mix->lock(2) w=1 flushing) (ifile mix->sync w=1 flushing) | dirtyscattered=2 lock=2 dirfrag=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x55b7e6612580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2024-09-17T04:34:30.331 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:34:30.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.449+0000 7fa8b79da640 1 -- 172.21.15.37:0/3192010285 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa8b00b1e00 msgr2=0x7fa8b00b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:30.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.449+0000 7fa8b79da640 1 --2- 172.21.15.37:0/3192010285 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa8b00b1e00 0x7fa8b00b2210 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4009830 tx=0x7fa8a402a480 comp rx=0 tx=0).stop 2024-09-17T04:34:30.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.449+0000 7fa8b79da640 1 -- 172.21.15.37:0/3192010285 shutdown_connections 2024-09-17T04:34:30.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.449+0000 7fa8b79da640 1 --2- 172.21.15.37:0/3192010285 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa8b00b1e00 0x7fa8b00b2210 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.449+0000 7fa8b79da640 1 --2- 172.21.15.37:0/3192010285 >> v2:172.21.15.92:3300/0 conn(0x7fa8b00b77a0 0x7fa8b00cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.449+0000 7fa8b79da640 1 -- 172.21.15.37:0/3192010285 >> 172.21.15.37:0/3192010285 conn(0x7fa8b00d2470 msgr2=0x7fa8b00d2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:30.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.449+0000 7fa8b79da640 1 -- 172.21.15.37:0/3192010285 shutdown_connections 2024-09-17T04:34:30.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.449+0000 7fa8b79da640 1 -- 172.21.15.37:0/3192010285 wait complete. 2024-09-17T04:34:30.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.451+0000 7fa8b79da640 1 Processor -- start 2024-09-17T04:34:30.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.451+0000 7fa8b79da640 1 -- start start 2024-09-17T04:34:30.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.452+0000 7fa8b79da640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8b00b1e00 0x7fa8b015a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:30.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.452+0000 7fa8b79da640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa8b00b77a0 0x7fa8b0071250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:30.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.452+0000 7fa8b79da640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8b00cea60 0x7fa8b0071790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:30.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.452+0000 7fa8b79da640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b00b28d0 con 0x7fa8b00b1e00 2024-09-17T04:34:30.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.452+0000 7fa8b79da640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b00b2750 con 0x7fa8b00cea60 2024-09-17T04:34:30.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.452+0000 7fa8b79da640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fa8b00b2f70 con 0x7fa8b00b77a0 2024-09-17T04:34:30.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.452+0000 7fa8b5776640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8b00b1e00 0x7fa8b015a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:30.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.452+0000 7fa8b5776640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8b00b1e00 0x7fa8b015a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48300/0 (socket says 172.21.15.37:48300) 2024-09-17T04:34:30.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.452+0000 7fa8b5776640 1 -- 172.21.15.37:0/3822565941 learned_addr learned my addr 172.21.15.37:0/3822565941 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:30.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.452+0000 7fa8b5f77640 1 --2- 172.21.15.37:0/3822565941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8b00cea60 0x7fa8b0071790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:30.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.452+0000 7fa8b4f75640 1 --2- 172.21.15.37:0/3822565941 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa8b00b77a0 0x7fa8b0071250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:30.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.452+0000 7fa8b5776640 1 -- 172.21.15.37:0/3822565941 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa8b00b77a0 msgr2=0x7fa8b0071250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:30.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.452+0000 7fa8b5776640 1 --2- 172.21.15.37:0/3822565941 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa8b00b77a0 0x7fa8b0071250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.452+0000 7fa8b5776640 1 -- 172.21.15.37:0/3822565941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8b00cea60 msgr2=0x7fa8b0071790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:30.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.452+0000 7fa8b5776640 1 --2- 172.21.15.37:0/3822565941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8b00cea60 0x7fa8b0071790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.452+0000 7fa8b5776640 1 -- 172.21.15.37:0/3822565941 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b015d600 con 0x7fa8b00b1e00 2024-09-17T04:34:30.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.453+0000 7fa8b5776640 1 --2- 172.21.15.37:0/3822565941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8b00b1e00 0x7fa8b015a5d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa8a000ed00 tx=0x7fa8a000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:30.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.453+0000 7fa8adffb640 1 -- 172.21.15.37:0/3822565941 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0004d40 con 0x7fa8b00b1e00 2024-09-17T04:34:30.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.453+0000 7fa8adffb640 1 -- 172.21.15.37:0/3822565941 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8a0003970 con 0x7fa8b00b1e00 2024-09-17T04:34:30.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.453+0000 7fa8b79da640 1 -- 172.21.15.37:0/3822565941 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8b015d8e0 con 0x7fa8b00b1e00 2024-09-17T04:34:30.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.453+0000 7fa8adffb640 1 -- 172.21.15.37:0/3822565941 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0003da0 con 0x7fa8b00b1e00 2024-09-17T04:34:30.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.453+0000 7fa8b79da640 1 -- 172.21.15.37:0/3822565941 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8b015dea0 con 0x7fa8b00b1e00 2024-09-17T04:34:30.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.454+0000 7fa8b79da640 1 -- 172.21.15.37:0/3822565941 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8b00b2210 con 0x7fa8b00b1e00 2024-09-17T04:34:30.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.455+0000 7fa8adffb640 1 -- 172.21.15.37:0/3822565941 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa8a0004020 con 0x7fa8b00b1e00 2024-09-17T04:34:30.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.456+0000 7fa8adffb640 1 --2- 172.21.15.37:0/3822565941 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa89c078bc0 0x7fa89c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:30.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.456+0000 7fa8adffb640 1 -- 172.21.15.37:0/3822565941 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7fa8a009bb90 con 0x7fa8b00b1e00 2024-09-17T04:34:30.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.456+0000 7fa8b4f75640 1 --2- 172.21.15.37:0/3822565941 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa89c078bc0 0x7fa89c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:30.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.456+0000 7fa8b4f75640 1 --2- 172.21.15.37:0/3822565941 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa89c078bc0 0x7fa89c07b0d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4034000 tx=0x7fa8a4004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:30.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.460+0000 7fa8adffb640 1 -- 172.21.15.37:0/3822565941 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa8a0063db0 con 0x7fa8b00b1e00 2024-09-17T04:34:30.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.682+0000 7fa8b79da640 1 -- 172.21.15.37:0/3822565941 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8b00b7210 con 0x7fa8b00b1e00 2024-09-17T04:34:30.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.683+0000 7fa8adffb640 1 -- 172.21.15.37:0/3822565941 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+6522 (secure 0 0 0) 0x7fa8a00636b0 con 0x7fa8b00b1e00 2024-09-17T04:34:30.683 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:30.683 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"btime":"2024-09-17T04:34:28:244441+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/199192833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":199192833},{"type":"v1","addr":"172.21.15.37:6833","nonce":199192833}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4832,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/4098913371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4098913371},{"type":"v1","addr":"172.21.15.92:6836","nonce":4098913371}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/383658044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":383658044},{"type":"v1","addr":"172.21.15.92:6837","nonce":383658044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:59.490265+0000","modified":"2024-09-17T04:34:15.700096+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2},{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:34:17.928780+0000","modified":"2024-09-17T04:34:28.244429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5208":{"gid":5208,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6835/4051839052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":4051839052},{"type":"v1","addr":"172.21.15.37:6835","nonce":4051839052}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":6,"enabled":true,"fs_name":"recovery_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5208,"qdb_cluster":[5208]},"id":3}]} 2024-09-17T04:34:30.683 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-09-17T04:34:30.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.685+0000 7fa8b79da640 1 -- 172.21.15.37:0/3822565941 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa89c078bc0 msgr2=0x7fa89c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:30.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.685+0000 7fa8b79da640 1 --2- 172.21.15.37:0/3822565941 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa89c078bc0 0x7fa89c07b0d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4034000 tx=0x7fa8a4004730 comp rx=0 tx=0).stop 2024-09-17T04:34:30.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.685+0000 7fa8b79da640 1 -- 172.21.15.37:0/3822565941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8b00b1e00 msgr2=0x7fa8b015a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:30.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.685+0000 7fa8b79da640 1 --2- 172.21.15.37:0/3822565941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8b00b1e00 0x7fa8b015a5d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa8a000ed00 tx=0x7fa8a000c6a0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.685+0000 7fa8b79da640 1 -- 172.21.15.37:0/3822565941 shutdown_connections 2024-09-17T04:34:30.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.685+0000 7fa8b79da640 1 --2- 172.21.15.37:0/3822565941 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa89c078bc0 0x7fa89c07b0d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.685+0000 7fa8b79da640 1 --2- 172.21.15.37:0/3822565941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8b00cea60 0x7fa8b0071790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.685+0000 7fa8b79da640 1 --2- 172.21.15.37:0/3822565941 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa8b00b77a0 0x7fa8b0071250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.685+0000 7fa8b79da640 1 --2- 172.21.15.37:0/3822565941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8b00b1e00 0x7fa8b015a5d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.685+0000 7fa8b79da640 1 -- 172.21.15.37:0/3822565941 >> 172.21.15.37:0/3822565941 conn(0x7fa8b00d2470 msgr2=0x7fa8b00b5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:30.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.685+0000 7fa8b79da640 1 -- 172.21.15.37:0/3822565941 shutdown_connections 2024-09-17T04:34:30.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.685+0000 7fa8b79da640 1 -- 172.21.15.37:0/3822565941 wait complete. 2024-09-17T04:34:30.697 INFO:tasks.cephfs.test_recovery_pool:{ "btime": "2024-09-17T04:34:28:244441+0000", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 2, "epoch": 27, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-17T04:33:59.490265+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 18, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [ 0 ], "flags": 19, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": false, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": {}, "last_failure": 0, "last_failure_osd_epoch": 34, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-17T04:34:15.700096+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": {} } }, { "id": 3, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-17T04:34:17.928780+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 27, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "recovery_fs", "in": [ 0 ], "info": { "gid_5208": { "addr": "172.21.15.37:6835/4051839052", "addrs": { "addrvec": [ { "addr": "172.21.15.37:6834", "nonce": 4051839052, "type": "v2" }, { "addr": "172.21.15.37:6835", "nonce": 4051839052, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540719139924082687, "flags": 0, "gid": 5208, "incarnation": 24, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 6 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 6, "modified": "2024-09-17T04:34:28.244429+0000", "qdb_cluster": [ 5208 ], "qdb_leader": 5208, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5208 } } } ], "standbys": [ { "addr": "172.21.15.37:6833/199192833", "addrs": { "addrvec": [ { "addr": "172.21.15.37:6832", "nonce": 199192833, "type": "v2" }, { "addr": "172.21.15.37:6833", "nonce": 199192833, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 19, "export_targets": [], "features": 4540719139924082687, "flags": 0, "gid": 4826, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.92:6836/4098913371", "addrs": { "addrvec": [ { "addr": "172.21.15.92:6834", "nonce": 4098913371, "type": "v2" }, { "addr": "172.21.15.92:6836", "nonce": 4098913371, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 19, "export_targets": [], "features": 4540719139924082687, "flags": 0, "gid": 4832, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.92:6837/383658044", "addrs": { "addrvec": [ { "addr": "172.21.15.92:6835", "nonce": 383658044, "type": "v2" }, { "addr": "172.21.15.92:6837", "nonce": 383658044, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 19, "export_targets": [], "features": 4540719139924082687, "flags": 0, "gid": 5068, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-17T04:34:30.697 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T04:34:30.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.813+0000 7fd93e5e4640 1 -- 172.21.15.37:0/3984476402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9380b1e00 msgr2=0x7fd9380b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:30.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.813+0000 7fd93e5e4640 1 --2- 172.21.15.37:0/3984476402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9380b1e00 0x7fd9380b2210 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd92c005fd0 tx=0x7fd92c02a5f0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.813+0000 7fd93e5e4640 1 -- 172.21.15.37:0/3984476402 shutdown_connections 2024-09-17T04:34:30.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.813+0000 7fd93e5e4640 1 --2- 172.21.15.37:0/3984476402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9380b1e00 0x7fd9380b2210 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.813+0000 7fd93e5e4640 1 -- 172.21.15.37:0/3984476402 >> 172.21.15.37:0/3984476402 conn(0x7fd9380d2470 msgr2=0x7fd9380d2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:30.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.813+0000 7fd93e5e4640 1 -- 172.21.15.37:0/3984476402 shutdown_connections 2024-09-17T04:34:30.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.813+0000 7fd93e5e4640 1 -- 172.21.15.37:0/3984476402 wait complete. 2024-09-17T04:34:30.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.815+0000 7fd93e5e4640 1 Processor -- start 2024-09-17T04:34:30.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.815+0000 7fd93e5e4640 1 -- start start 2024-09-17T04:34:30.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.816+0000 7fd93e5e4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9380b1e00 0x7fd9380cc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:30.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.816+0000 7fd93e5e4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9380b77a0 0x7fd938164910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:30.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.816+0000 7fd93e5e4640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd9380cea60 0x7fd9381f6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:30.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.816+0000 7fd93e5e4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd9380b30f0 con 0x7fd9380b1e00 2024-09-17T04:34:30.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.816+0000 7fd93e5e4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd9380b2f70 con 0x7fd9380b77a0 2024-09-17T04:34:30.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.816+0000 7fd93e5e4640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fd938168150 con 0x7fd9380cea60 2024-09-17T04:34:30.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.817+0000 7fd937fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9380b1e00 0x7fd9380cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:30.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.817+0000 7fd937fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9380b1e00 0x7fd9380cc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48314/0 (socket says 172.21.15.37:48314) 2024-09-17T04:34:30.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.817+0000 7fd937fff640 1 -- 172.21.15.37:0/3053789571 learned_addr learned my addr 172.21.15.37:0/3053789571 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:30.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.817+0000 7fd9377fe640 1 --2- 172.21.15.37:0/3053789571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9380b77a0 0x7fd938164910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:30.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.817+0000 7fd93cb81640 1 --2- 172.21.15.37:0/3053789571 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd9380cea60 0x7fd9381f6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:30.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.817+0000 7fd937fff640 1 -- 172.21.15.37:0/3053789571 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd9380cea60 msgr2=0x7fd9381f6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:30.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.817+0000 7fd937fff640 1 --2- 172.21.15.37:0/3053789571 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd9380cea60 0x7fd9381f6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.817+0000 7fd937fff640 1 -- 172.21.15.37:0/3053789571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9380b77a0 msgr2=0x7fd938164910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:30.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.817+0000 7fd937fff640 1 --2- 172.21.15.37:0/3053789571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9380b77a0 0x7fd938164910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:30.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.817+0000 7fd937fff640 1 -- 172.21.15.37:0/3053789571 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9380cc9c0 con 0x7fd9380b1e00 2024-09-17T04:34:30.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.817+0000 7fd93cb81640 1 --2- 172.21.15.37:0/3053789571 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd9380cea60 0x7fd9381f6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:30.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.817+0000 7fd937fff640 1 --2- 172.21.15.37:0/3053789571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9380b1e00 0x7fd9380cc480 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd92800c400 tx=0x7fd92800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:30.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.817+0000 7fd934ff9640 1 -- 172.21.15.37:0/3053789571 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd928017070 con 0x7fd9380b1e00 2024-09-17T04:34:30.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.817+0000 7fd934ff9640 1 -- 172.21.15.37:0/3053789571 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9280053c0 con 0x7fd9380b1e00 2024-09-17T04:34:30.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.817+0000 7fd93e5e4640 1 -- 172.21.15.37:0/3053789571 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9380afca0 con 0x7fd9380b1e00 2024-09-17T04:34:30.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.817+0000 7fd934ff9640 1 -- 172.21.15.37:0/3053789571 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9280038a0 con 0x7fd9380b1e00 2024-09-17T04:34:30.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.817+0000 7fd93e5e4640 1 -- 172.21.15.37:0/3053789571 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9380b0260 con 0x7fd9380b1e00 2024-09-17T04:34:30.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.818+0000 7fd93e5e4640 1 -- 172.21.15.37:0/3053789571 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9380b2210 con 0x7fd9380b1e00 2024-09-17T04:34:30.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.819+0000 7fd934ff9640 1 -- 172.21.15.37:0/3053789571 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd92801a070 con 0x7fd9380b1e00 2024-09-17T04:34:30.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.820+0000 7fd934ff9640 1 --2- 172.21.15.37:0/3053789571 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd924078b90 0x7fd92407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:30.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.820+0000 7fd934ff9640 1 -- 172.21.15.37:0/3053789571 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7fd92809a2b0 con 0x7fd9380b1e00 2024-09-17T04:34:30.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.820+0000 7fd9377fe640 1 --2- 172.21.15.37:0/3053789571 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd924078b90 0x7fd92407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:30.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.821+0000 7fd9377fe640 1 --2- 172.21.15.37:0/3053789571 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd924078b90 0x7fd92407b0a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd92c033000 tx=0x7fd92c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:30.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:30.825+0000 7fd934ff9640 1 -- 172.21.15.37:0/3053789571 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd928062560 con 0x7fd9380b1e00 2024-09-17T04:34:31.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:31.046+0000 7fd93e5e4640 1 -- 172.21.15.37:0/3053789571 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd938164e50 con 0x7fd9380b1e00 2024-09-17T04:34:31.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:31.047+0000 7fd934ff9640 1 -- 172.21.15.37:0/3053789571 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+156 (secure 0 0 0) 0x7fd928061e60 con 0x7fd9380b1e00 2024-09-17T04:34:31.047 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T04:34:31.047 INFO:teuthology.orchestra.run.smithi037.stdout:name: recovery_fs, metadata pool: recovery_fs_metadata, data pools: [cephfs_data ] 2024-09-17T04:34:31.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:31.049+0000 7fd93e5e4640 1 -- 172.21.15.37:0/3053789571 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd924078b90 msgr2=0x7fd92407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:31.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:31.049+0000 7fd93e5e4640 1 --2- 172.21.15.37:0/3053789571 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd924078b90 0x7fd92407b0a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd92c033000 tx=0x7fd92c004730 comp rx=0 tx=0).stop 2024-09-17T04:34:31.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:31.049+0000 7fd93e5e4640 1 -- 172.21.15.37:0/3053789571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9380b1e00 msgr2=0x7fd9380cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:31.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:31.049+0000 7fd93e5e4640 1 --2- 172.21.15.37:0/3053789571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9380b1e00 0x7fd9380cc480 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd92800c400 tx=0x7fd92800c8c0 comp rx=0 tx=0).stop 2024-09-17T04:34:31.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:31.049+0000 7fd93e5e4640 1 -- 172.21.15.37:0/3053789571 shutdown_connections 2024-09-17T04:34:31.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:31.049+0000 7fd93e5e4640 1 --2- 172.21.15.37:0/3053789571 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd924078b90 0x7fd92407b0a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:31.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:31.049+0000 7fd93e5e4640 1 --2- 172.21.15.37:0/3053789571 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd9380cea60 0x7fd9381f6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:31.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:31.049+0000 7fd93e5e4640 1 --2- 172.21.15.37:0/3053789571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9380b77a0 0x7fd938164910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:31.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:31.049+0000 7fd93e5e4640 1 --2- 172.21.15.37:0/3053789571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9380b1e00 0x7fd9380cc480 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:31.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:31.049+0000 7fd93e5e4640 1 -- 172.21.15.37:0/3053789571 >> 172.21.15.37:0/3053789571 conn(0x7fd9380d2470 msgr2=0x7fd938072820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:31.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:31.049+0000 7fd93e5e4640 1 -- 172.21.15.37:0/3053789571 shutdown_connections 2024-09-17T04:34:31.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:31.049+0000 7fd93e5e4640 1 -- 172.21.15.37:0/3053789571 wait complete. 2024-09-17T04:34:31.060 INFO:tasks.cephfs.mount:Mounting Ceph FS recovery_fs; just confirmed its presence on cluster 2024-09-17T04:34:31.060 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T04:34:31.060 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-09-17T04:34:31.060 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T04:34:31.060 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T04:34:31.060 INFO:tasks.cephfs.mount:self.cephfs_name = recovery_fs 2024-09-17T04:34:31.060 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T04:34:31.060 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T04:34:31.060 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:34:31.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:31.061 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-09-17T04:34:31.077 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T04:34:31.077 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T04:34:31.077 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T04:34:31.077 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:31.077 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-09-17T04:34:31.077 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:31.077 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:34:31.078 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:34:31.078 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-09-17T04:34:31.078 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:34:31.078 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:34:31.078 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-09-17T04:34:31.078 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T04:34:31.078 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:34:31.078 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-09-17T04:34:31.078 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-17T04:34:31.078 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42681sec preferred_lft 42681sec 2024-09-17T04:34:31.078 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-09-17T04:34:31.078 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:31.078 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T04:34:31.078 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:34:31.079 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-09-17T04:34:31.079 INFO:teuthology.orchestra.run.smithi037.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T04:34:31.079 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 12:28:aa:73:a5:d1 brd ff:ff:ff:ff:ff:ff 2024-09-17T04:34:31.079 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T04:34:31.079 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:31.079 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::8fb:ceff:fe9f:ed7d/64 scope link 2024-09-17T04:34:31.079 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:31.079 INFO:teuthology.orchestra.run.smithi037.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:34:31.079 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether d6:3e:49:21:b4:0b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:34:31.079 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::d43e:49ff:fe21:b40b/64 scope link 2024-09-17T04:34:31.079 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:31.079 INFO:teuthology.orchestra.run.smithi037.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T04:34:31.079 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 12:28:aa:73:a5:d1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:34:31.080 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::1028:aaff:fe73:a5d1/64 scope link 2024-09-17T04:34:31.080 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-17T04:34:31.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:31.080 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T04:34:31.142 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-09-17T04:34:31.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:31.144 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-09-17T04:34:31.200 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-09-17T04:34:31.200 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T04:34:31.201 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-17T04:34:31.201 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-17T04:34:31.201 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T04:34:31.201 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T04:34:31.201 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-17T04:34:31.201 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T04:34:31.202 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T04:34:31.202 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T04:34:31.202 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-17T04:34:31.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T04:34:31.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:34:31.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:31.330 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-09-17T04:34:31.346 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T04:34:31.346 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T04:34:31.346 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:34:31.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:31.346 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-09-17T04:34:31.411 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-17T04:34:31.412 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:34:31.412 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T04:34:31.467 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T04:34:31.468 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T04:34:31.468 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:34:31.523 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-09-17T04:34:31.590 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T04:34:31.647 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-09-17T04:34:31.647 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-09-17T04:34:31.647 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-09-17T04:34:31.647 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-09-17T04:34:31.647 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-09-17T04:34:31.647 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-09-17T04:34:31.647 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-09-17T04:34:31.647 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-09-17T04:34:31.647 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-09-17T04:34:31.647 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-09-17T04:34:31.647 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-09-17T04:34:31.648 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/selinux 2024-09-17T04:34:31.648 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T04:34:31.648 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-09-17T04:34:31.648 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-09-17T04:34:31.648 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-09-17T04:34:31.648 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-09-17T04:34:31.648 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-09-17T04:34:31.648 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-09-17T04:34:31.648 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T04:34:31.648 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T04:34:31.648 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T04:34:31.648 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T04:34:31.648 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-09-17T04:34:31.648 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-09-17T04:34:31.649 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T04:34:31.649 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T04:34:31.649 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T04:34:31.649 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T04:34:31.649 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-09-17T04:34:31.649 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:34:31.649 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:34:31.649 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:34:31.649 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:34:31.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:31.649 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-09-17T04:34:31.704 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T04:34:31.704 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_fs=recovery_fs) 2024-09-17T04:34:31.710 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-09-17T04:34:31.710+0000 7f55141d9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T04:34:31.712 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-09-17T04:34:31.737 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T04:34:31.796 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-09-17T04:34:31.796 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-09-17T04:34:31.796 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-09-17T04:34:31.796 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-09-17T04:34:31.796 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-09-17T04:34:31.796 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-09-17T04:34:31.797 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-09-17T04:34:31.797 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-09-17T04:34:31.797 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-09-17T04:34:31.797 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-09-17T04:34:31.797 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-09-17T04:34:31.797 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/selinux 2024-09-17T04:34:31.797 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T04:34:31.797 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-09-17T04:34:31.797 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-09-17T04:34:31.797 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-09-17T04:34:31.797 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-09-17T04:34:31.797 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-09-17T04:34:31.797 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-09-17T04:34:31.797 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T04:34:31.798 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T04:34:31.798 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T04:34:31.798 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T04:34:31.798 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-09-17T04:34:31.798 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-09-17T04:34:31.798 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T04:34:31.798 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T04:34:31.798 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T04:34:31.798 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T04:34:31.798 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-09-17T04:34:31.798 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:34:31.798 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:34:31.798 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:34:31.799 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:34:31.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:31.799 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-09-17T04:34:31.806 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:2024-09-17T04:34:31.806+0000 7fc045544480 -1 init, newargv = 0x7fc028003020 newargc=15 2024-09-17T04:34:31.806 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[50943]: starting ceph client 2024-09-17T04:34:31.812 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[50943]: starting fuse 2024-09-17T04:34:31.823 INFO:teuthology.orchestra.run.smithi037.stdout:42 2024-09-17T04:34:31.823 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-17T04:34:31.823 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-09-17T04:34:31.823 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-09-17T04:34:31.823 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-09-17T04:34:31.823 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-17T04:34:31.823 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-09-17T04:34:31.824 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-17T04:34:31.824 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-09-17T04:34:31.824 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-17T04:34:31.824 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-09-17T04:34:31.824 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-17T04:34:31.824 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-17T04:34:31.824 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-09-17T04:34:31.824 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-09-17T04:34:31.824 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-09-17T04:34:31.824 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-09-17T04:34:31.824 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-17T04:34:31.824 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-09-17T04:34:31.824 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T04:34:31.825 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T04:34:31.825 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T04:34:31.825 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-09-17T04:34:31.825 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-09-17T04:34:31.825 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-09-17T04:34:31.825 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T04:34:31.825 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-17T04:34:31.825 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.0")) 2024-09-17T04:34:31.825 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-17T04:34:31.943 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.0.50943.asok 2024-09-17T04:34:31.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:34:31.954 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50943.asok 2024-09-17T04:34:31.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:31.955 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50943.asok status 2024-09-17T04:34:32.101 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T04:34:32.101 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-09-17T04:34:32.101 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T04:34:32.101 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T04:34:32.101 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "0", 2024-09-17T04:34:32.101 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-09-17T04:34:32.101 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-17T04:34:32.101 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "50943", 2024-09-17T04:34:32.101 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-09-17T04:34:32.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:34:32.102 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-09-17T04:34:32.102 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-09-17T04:34:32.102 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 5487, 2024-09-17T04:34:32.102 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-09-17T04:34:32.102 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-09-17T04:34:32.102 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-09-17T04:34:32.102 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 5487 2024-09-17T04:34:32.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:34:32.102 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-09-17T04:34:32.102 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-09-17T04:34:32.102 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-09-17T04:34:32.102 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 635250385 2024-09-17T04:34:32.102 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T04:34:32.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:34:32.103 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-09-17T04:34:32.103 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-09-17T04:34:32.103 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-09-17T04:34:32.103 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 635250385 2024-09-17T04:34:32.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T04:34:32.103 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.5487 192.168.0.1:0/635250385", 2024-09-17T04:34:32.103 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.1:0/635250385", 2024-09-17T04:34:32.103 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-09-17T04:34:32.103 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 27, 2024-09-17T04:34:32.103 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 41, 2024-09-17T04:34:32.103 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-09-17T04:34:32.103 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-09-17T04:34:32.103 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "recovery_fs" 2024-09-17T04:34:32.103 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T04:34:32.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:32.110 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-09-17T04:34:32.110 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-17T04:34:32.126 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-09-17T04:34:32.126 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-17T04:34:32.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:32.127 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T04:34:32.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:32.195 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir) 2024-09-17T04:34:32.256 INFO:teuthology.orchestra.run.smithi037.stdout:sixmegs 2024-09-17T04:34:32.257 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-09-17T04:34:32.257 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-09-17T04:34:32.257 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-09-17T04:34:32.257 DEBUG:teuthology.orchestra.run.smithi037:> import json 2024-09-17T04:34:32.257 DEBUG:teuthology.orchestra.run.smithi037:> import sys 2024-09-17T04:34:32.257 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-17T04:34:32.257 DEBUG:teuthology.orchestra.run.smithi037:> try: 2024-09-17T04:34:32.257 DEBUG:teuthology.orchestra.run.smithi037:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2024-09-17T04:34:32.257 DEBUG:teuthology.orchestra.run.smithi037:> except OSError as e: 2024-09-17T04:34:32.258 DEBUG:teuthology.orchestra.run.smithi037:> sys.exit(e.errno) 2024-09-17T04:34:32.258 DEBUG:teuthology.orchestra.run.smithi037:> 2024-09-17T04:34:32.258 DEBUG:teuthology.orchestra.run.smithi037:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-09-17T04:34:32.258 DEBUG:teuthology.orchestra.run.smithi037:> print(json.dumps( 2024-09-17T04:34:32.258 DEBUG:teuthology.orchestra.run.smithi037:> dict([(a, getattr(s, a)) for a in attrs]), 2024-09-17T04:34:32.258 DEBUG:teuthology.orchestra.run.smithi037:> indent=2)) 2024-09-17T04:34:32.258 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-09-17T04:34:32.366 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T04:34:32.366 INFO:teuthology.orchestra.run.smithi037.stdout: "st_mode": 33088, 2024-09-17T04:34:32.367 INFO:teuthology.orchestra.run.smithi037.stdout: "st_ino": 1099511627777, 2024-09-17T04:34:32.367 INFO:teuthology.orchestra.run.smithi037.stdout: "st_dev": 42, 2024-09-17T04:34:32.367 INFO:teuthology.orchestra.run.smithi037.stdout: "st_nlink": 1, 2024-09-17T04:34:32.367 INFO:teuthology.orchestra.run.smithi037.stdout: "st_uid": 0, 2024-09-17T04:34:32.367 INFO:teuthology.orchestra.run.smithi037.stdout: "st_gid": 0, 2024-09-17T04:34:32.367 INFO:teuthology.orchestra.run.smithi037.stdout: "st_size": 6291456, 2024-09-17T04:34:32.368 INFO:teuthology.orchestra.run.smithi037.stdout: "st_atime": 1726547655.0, 2024-09-17T04:34:32.368 INFO:teuthology.orchestra.run.smithi037.stdout: "st_mtime": 1726547655.0, 2024-09-17T04:34:32.368 INFO:teuthology.orchestra.run.smithi037.stdout: "st_ctime": 1726547655.0 2024-09-17T04:34:32.368 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T04:34:32.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T04:34:32.375 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T04:34:32.375 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:34:32.455 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T04:34:32.500 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T04:34:32.501 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:34:32.501 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:34:32.501 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:34:32.517 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:34:32.518 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:34:32.574 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T04:34:32.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.732+0000 7fdcab2f9640 1 -- 172.21.15.37:0/2865051010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdca40cd1b0 msgr2=0x7fdca40cd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:32.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.732+0000 7fdcab2f9640 1 --2- 172.21.15.37:0/2865051010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdca40cd1b0 0x7fdca40cd5c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdca40b1e90 tx=0x7fdc9c02aac0 comp rx=0 tx=0).stop 2024-09-17T04:34:32.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.732+0000 7fdcab2f9640 1 -- 172.21.15.37:0/2865051010 shutdown_connections 2024-09-17T04:34:32.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.732+0000 7fdcab2f9640 1 --2- 172.21.15.37:0/2865051010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdca40cd1b0 0x7fdca40cd5c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:32.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.732+0000 7fdcab2f9640 1 --2- 172.21.15.37:0/2865051010 >> v2:172.21.15.92:3300/0 conn(0x7fdca40b74f0 0x7fdca40b7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:32.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.732+0000 7fdcab2f9640 1 -- 172.21.15.37:0/2865051010 >> 172.21.15.37:0/2865051010 conn(0x7fdca40d2460 msgr2=0x7fdca40d2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:32.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.732+0000 7fdcab2f9640 1 -- 172.21.15.37:0/2865051010 shutdown_connections 2024-09-17T04:34:32.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.732+0000 7fdcab2f9640 1 -- 172.21.15.37:0/2865051010 wait complete. 2024-09-17T04:34:32.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.734+0000 7fdcab2f9640 1 Processor -- start 2024-09-17T04:34:32.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.734+0000 7fdcab2f9640 1 -- start start 2024-09-17T04:34:32.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.735+0000 7fdcab2f9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdca4072a90 0x7fdca414d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:32.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.735+0000 7fdcab2f9640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdca40b74f0 0x7fdca40c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:32.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.735+0000 7fdcab2f9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdca40cd1b0 0x7fdca40c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:32.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.735+0000 7fdcab2f9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdca40cdc80 con 0x7fdca4072a90 2024-09-17T04:34:32.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.735+0000 7fdcab2f9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdca40cdb00 con 0x7fdca40cd1b0 2024-09-17T04:34:32.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.735+0000 7fdcab2f9640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fdca4161100 con 0x7fdca40b74f0 2024-09-17T04:34:32.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.735+0000 7fdca9095640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdca4072a90 0x7fdca414d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:32.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.735+0000 7fdca9095640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdca4072a90 0x7fdca414d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48330/0 (socket says 172.21.15.37:48330) 2024-09-17T04:34:32.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.735+0000 7fdca9095640 1 -- 172.21.15.37:0/2683150681 learned_addr learned my addr 172.21.15.37:0/2683150681 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:32.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.736+0000 7fdca9095640 1 -- 172.21.15.37:0/2683150681 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdca40b74f0 msgr2=0x7fdca40c7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:32.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.736+0000 7fdca8894640 1 --2- 172.21.15.37:0/2683150681 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdca40b74f0 0x7fdca40c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:32.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.736+0000 7fdca9896640 1 --2- 172.21.15.37:0/2683150681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdca40cd1b0 0x7fdca40c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:32.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.736+0000 7fdca9095640 1 --2- 172.21.15.37:0/2683150681 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdca40b74f0 0x7fdca40c7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:32.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.736+0000 7fdca9095640 1 -- 172.21.15.37:0/2683150681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdca40cd1b0 msgr2=0x7fdca40c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:32.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.736+0000 7fdca9095640 1 --2- 172.21.15.37:0/2683150681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdca40cd1b0 0x7fdca40c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:32.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.736+0000 7fdca9095640 1 -- 172.21.15.37:0/2683150681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca41562d0 con 0x7fdca4072a90 2024-09-17T04:34:32.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.736+0000 7fdca8894640 1 --2- 172.21.15.37:0/2683150681 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdca40b74f0 0x7fdca40c7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:32.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.736+0000 7fdca9896640 1 --2- 172.21.15.37:0/2683150681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdca40cd1b0 0x7fdca40c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:32.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.736+0000 7fdca9095640 1 --2- 172.21.15.37:0/2683150681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdca4072a90 0x7fdca414d800 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fdca000c390 tx=0x7fdca000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:32.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.736+0000 7fdc99ffb640 1 -- 172.21.15.37:0/2683150681 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca0017070 con 0x7fdca4072a90 2024-09-17T04:34:32.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.736+0000 7fdc99ffb640 1 -- 172.21.15.37:0/2683150681 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdca00035f0 con 0x7fdca4072a90 2024-09-17T04:34:32.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.736+0000 7fdc99ffb640 1 -- 172.21.15.37:0/2683150681 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca0003a20 con 0x7fdca4072a90 2024-09-17T04:34:32.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.736+0000 7fdcab2f9640 1 -- 172.21.15.37:0/2683150681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca40b2f40 con 0x7fdca4072a90 2024-09-17T04:34:32.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.736+0000 7fdcab2f9640 1 -- 172.21.15.37:0/2683150681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca40b31e0 con 0x7fdca4072a90 2024-09-17T04:34:32.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.737+0000 7fdcab2f9640 1 -- 172.21.15.37:0/2683150681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdca40b7900 con 0x7fdca4072a90 2024-09-17T04:34:32.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.738+0000 7fdc99ffb640 1 -- 172.21.15.37:0/2683150681 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdca001a070 con 0x7fdca4072a90 2024-09-17T04:34:32.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.739+0000 7fdc99ffb640 1 --2- 172.21.15.37:0/2683150681 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fdc88078b90 0x7fdc8807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:32.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.739+0000 7fdc99ffb640 1 -- 172.21.15.37:0/2683150681 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7fdca009a220 con 0x7fdca4072a90 2024-09-17T04:34:32.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.739+0000 7fdca8894640 1 --2- 172.21.15.37:0/2683150681 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fdc88078b90 0x7fdc8807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:32.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.739+0000 7fdca8894640 1 --2- 172.21.15.37:0/2683150681 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fdc88078b90 0x7fdc8807b0a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdca40cc740 tx=0x7fdc94005dd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:32.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.743+0000 7fdc99ffb640 1 -- 172.21.15.37:0/2683150681 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fdca00624d0 con 0x7fdca4072a90 2024-09-17T04:34:32.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.940+0000 7fdcab2f9640 1 -- 172.21.15.37:0/2683150681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdca406a150 con 0x7fdca4072a90 2024-09-17T04:34:32.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.941+0000 7fdc99ffb640 1 -- 172.21.15.37:0/2683150681 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v41) ==== 80+0+122 (secure 0 0 0) 0x7fdca0061dd0 con 0x7fdca4072a90 2024-09-17T04:34:32.941 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6835/1358600605 2024-09-18T04:34:15.699884+0000 2024-09-17T04:34:32.941 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6833/1358600605 2024-09-18T04:34:15.699884+0000 2024-09-17T04:34:32.941 INFO:teuthology.orchestra.run.smithi037.stderr:listed 2 entries 2024-09-17T04:34:32.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.943+0000 7fdcab2f9640 1 -- 172.21.15.37:0/2683150681 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fdc88078b90 msgr2=0x7fdc8807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:32.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.943+0000 7fdcab2f9640 1 --2- 172.21.15.37:0/2683150681 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fdc88078b90 0x7fdc8807b0a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdca40cc740 tx=0x7fdc94005dd0 comp rx=0 tx=0).stop 2024-09-17T04:34:32.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.943+0000 7fdcab2f9640 1 -- 172.21.15.37:0/2683150681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdca4072a90 msgr2=0x7fdca414d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:32.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.943+0000 7fdcab2f9640 1 --2- 172.21.15.37:0/2683150681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdca4072a90 0x7fdca414d800 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fdca000c390 tx=0x7fdca000c850 comp rx=0 tx=0).stop 2024-09-17T04:34:32.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.943+0000 7fdcab2f9640 1 -- 172.21.15.37:0/2683150681 shutdown_connections 2024-09-17T04:34:32.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.943+0000 7fdcab2f9640 1 --2- 172.21.15.37:0/2683150681 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fdc88078b90 0x7fdc8807b0a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:32.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.943+0000 7fdcab2f9640 1 --2- 172.21.15.37:0/2683150681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdca40cd1b0 0x7fdca40c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:32.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.943+0000 7fdcab2f9640 1 --2- 172.21.15.37:0/2683150681 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdca40b74f0 0x7fdca40c7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:32.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.943+0000 7fdcab2f9640 1 --2- 172.21.15.37:0/2683150681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdca4072a90 0x7fdca414d800 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:32.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.943+0000 7fdcab2f9640 1 -- 172.21.15.37:0/2683150681 >> 172.21.15.37:0/2683150681 conn(0x7fdca40d2460 msgr2=0x7fdca40b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:32.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.943+0000 7fdcab2f9640 1 -- 172.21.15.37:0/2683150681 shutdown_connections 2024-09-17T04:34:32.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:32.943+0000 7fdcab2f9640 1 -- 172.21.15.37:0/2683150681 wait complete. 2024-09-17T04:34:32.954 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-09-17T04:34:32.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:32.954 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-17T04:34:32.978 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[50943]: fuse finished with error 0 and tester_r 0 2024-09-17T04:34:32.980 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-09-17T04:34:32.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:32.980 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T04:34:33.147 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:34:33.147 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:34:33.147 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:34:33.167 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056024,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497460k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=824 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497460k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:34:33.167 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:34:33.167 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:34:33.167 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:34:33.204 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17600 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:34:33.204 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T04:34:33.204 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:34:33.204 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:34:33.225 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056024,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497460k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=824 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497460k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:34:33.226 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:34:33.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.361+0000 7faf0b952640 1 -- 172.21.15.37:0/2939564876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf040ce600 msgr2=0x7faf040b1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:33.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.361+0000 7faf0b952640 1 --2- 172.21.15.37:0/2939564876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf040ce600 0x7faf040b1560 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7faf040d0df0 tx=0x7faef402aa30 comp rx=0 tx=0).stop 2024-09-17T04:34:33.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.361+0000 7faf0b952640 1 -- 172.21.15.37:0/2939564876 shutdown_connections 2024-09-17T04:34:33.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.361+0000 7faf0b952640 1 --2- 172.21.15.37:0/2939564876 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7faf040cfd30 0x7faf0415e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:33.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.361+0000 7faf0b952640 1 --2- 172.21.15.37:0/2939564876 >> v2:172.21.15.92:3300/0 conn(0x7faf040d1e50 0x7faf040cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:33.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.361+0000 7faf0b952640 1 --2- 172.21.15.37:0/2939564876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf040ce600 0x7faf040b1560 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:33.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.361+0000 7faf0b952640 1 -- 172.21.15.37:0/2939564876 >> 172.21.15.37:0/2939564876 conn(0x7faf040d1330 msgr2=0x7faf040d1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:33.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.361+0000 7faf0b952640 1 -- 172.21.15.37:0/2939564876 shutdown_connections 2024-09-17T04:34:33.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.362+0000 7faf0b952640 1 -- 172.21.15.37:0/2939564876 wait complete. 2024-09-17T04:34:33.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.363+0000 7faf0b952640 1 Processor -- start 2024-09-17T04:34:33.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.363+0000 7faf0b952640 1 -- start start 2024-09-17T04:34:33.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.365+0000 7faf0b952640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7faf040ce600 0x7faf041f4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:33.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.365+0000 7faf0b952640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf040cfd30 0x7faf04156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:33.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.365+0000 7faf0b952640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf040d1e50 0x7faf040cf020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:33.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.365+0000 7faf0b952640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faf0415eaa0 con 0x7faf040cfd30 2024-09-17T04:34:33.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.365+0000 7faf0b952640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faf0415e920 con 0x7faf040d1e50 2024-09-17T04:34:33.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.365+0000 7faf0b952640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7faf0415ec20 con 0x7faf040ce600 2024-09-17T04:34:33.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.365+0000 7faf08eed640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf040cfd30 0x7faf04156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:33.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.365+0000 7faf08eed640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf040cfd30 0x7faf04156940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48352/0 (socket says 172.21.15.37:48352) 2024-09-17T04:34:33.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.365+0000 7faf08eed640 1 -- 172.21.15.37:0/4196066322 learned_addr learned my addr 172.21.15.37:0/4196066322 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:33.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.365+0000 7faf096ee640 1 --2- 172.21.15.37:0/4196066322 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7faf040ce600 0x7faf041f4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:33.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.365+0000 7faf09eef640 1 --2- 172.21.15.37:0/4196066322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf040d1e50 0x7faf040cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:33.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.365+0000 7faf08eed640 1 -- 172.21.15.37:0/4196066322 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7faf040ce600 msgr2=0x7faf041f4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:33.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.365+0000 7faf08eed640 1 --2- 172.21.15.37:0/4196066322 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7faf040ce600 0x7faf041f4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:33.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.365+0000 7faf08eed640 1 -- 172.21.15.37:0/4196066322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf040d1e50 msgr2=0x7faf040cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:33.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.365+0000 7faf08eed640 1 --2- 172.21.15.37:0/4196066322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf040d1e50 0x7faf040cf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:33.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.365+0000 7faf08eed640 1 -- 172.21.15.37:0/4196066322 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf041f72b0 con 0x7faf040cfd30 2024-09-17T04:34:33.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.365+0000 7faf09eef640 1 --2- 172.21.15.37:0/4196066322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf040d1e50 0x7faf040cf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:33.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.365+0000 7faf08eed640 1 --2- 172.21.15.37:0/4196066322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf040cfd30 0x7faf04156940 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7faef800d780 tx=0x7faef800dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:33.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.365+0000 7faf01ffb640 1 -- 172.21.15.37:0/4196066322 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8017070 con 0x7faf040cfd30 2024-09-17T04:34:33.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.365+0000 7faf01ffb640 1 -- 172.21.15.37:0/4196066322 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faef800de90 con 0x7faf040cfd30 2024-09-17T04:34:33.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.365+0000 7faf0b952640 1 -- 172.21.15.37:0/4196066322 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf040b63e0 con 0x7faf040cfd30 2024-09-17T04:34:33.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.366+0000 7faf01ffb640 1 -- 172.21.15.37:0/4196066322 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef80099d0 con 0x7faf040cfd30 2024-09-17T04:34:33.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.366+0000 7faf0b952640 1 -- 172.21.15.37:0/4196066322 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf041f7870 con 0x7faf040cfd30 2024-09-17T04:34:33.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.367+0000 7faf0b952640 1 -- 172.21.15.37:0/4196066322 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf0415f140 con 0x7faf040cfd30 2024-09-17T04:34:33.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.367+0000 7faf01ffb640 1 -- 172.21.15.37:0/4196066322 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faef8009b70 con 0x7faf040cfd30 2024-09-17T04:34:33.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.368+0000 7faf01ffb640 1 --2- 172.21.15.37:0/4196066322 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7faef0078bc0 0x7faef007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:33.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.368+0000 7faf01ffb640 1 -- 172.21.15.37:0/4196066322 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7faef809b360 con 0x7faf040cfd30 2024-09-17T04:34:33.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.369+0000 7faf096ee640 1 --2- 172.21.15.37:0/4196066322 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7faef0078bc0 0x7faef007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:33.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.369+0000 7faf096ee640 1 --2- 172.21.15.37:0/4196066322 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7faef0078bc0 0x7faef007b0d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faef4035000 tx=0x7faef403b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:33.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.373+0000 7faf01ffb640 1 -- 172.21.15.37:0/4196066322 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7faef8020020 con 0x7faf040cfd30 2024-09-17T04:34:33.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.600+0000 7faf0b952640 1 -- 172.21.15.37:0/4196066322 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf0415dac0 con 0x7faf040cfd30 2024-09-17T04:34:33.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.600+0000 7faf01ffb640 1 -- 172.21.15.37:0/4196066322 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+6522 (secure 0 0 0) 0x7faef80634e0 con 0x7faf040cfd30 2024-09-17T04:34:33.601 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:33.601 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"btime":"2024-09-17T04:34:28:244441+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/199192833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":199192833},{"type":"v1","addr":"172.21.15.37:6833","nonce":199192833}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4832,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/4098913371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4098913371},{"type":"v1","addr":"172.21.15.92:6836","nonce":4098913371}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/383658044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":383658044},{"type":"v1","addr":"172.21.15.92:6837","nonce":383658044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:59.490265+0000","modified":"2024-09-17T04:34:15.700096+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2},{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:34:17.928780+0000","modified":"2024-09-17T04:34:28.244429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5208":{"gid":5208,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6835/4051839052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":4051839052},{"type":"v1","addr":"172.21.15.37:6835","nonce":4051839052}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":6,"enabled":true,"fs_name":"recovery_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5208,"qdb_cluster":[5208]},"id":3}]} 2024-09-17T04:34:33.601 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-09-17T04:34:33.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.603+0000 7faf0b952640 1 -- 172.21.15.37:0/4196066322 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7faef0078bc0 msgr2=0x7faef007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:33.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.603+0000 7faf0b952640 1 --2- 172.21.15.37:0/4196066322 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7faef0078bc0 0x7faef007b0d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faef4035000 tx=0x7faef403b040 comp rx=0 tx=0).stop 2024-09-17T04:34:33.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.603+0000 7faf0b952640 1 -- 172.21.15.37:0/4196066322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf040cfd30 msgr2=0x7faf04156940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:33.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.603+0000 7faf0b952640 1 --2- 172.21.15.37:0/4196066322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf040cfd30 0x7faf04156940 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7faef800d780 tx=0x7faef800dc40 comp rx=0 tx=0).stop 2024-09-17T04:34:33.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.603+0000 7faf0b952640 1 -- 172.21.15.37:0/4196066322 shutdown_connections 2024-09-17T04:34:33.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.603+0000 7faf0b952640 1 --2- 172.21.15.37:0/4196066322 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7faef0078bc0 0x7faef007b0d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:33.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.603+0000 7faf0b952640 1 --2- 172.21.15.37:0/4196066322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf040d1e50 0x7faf040cf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:33.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.603+0000 7faf0b952640 1 --2- 172.21.15.37:0/4196066322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf040cfd30 0x7faf04156940 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:33.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.603+0000 7faf0b952640 1 --2- 172.21.15.37:0/4196066322 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7faf040ce600 0x7faf041f4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:33.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.603+0000 7faf0b952640 1 -- 172.21.15.37:0/4196066322 >> 172.21.15.37:0/4196066322 conn(0x7faf040d1330 msgr2=0x7faf040718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:33.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.603+0000 7faf0b952640 1 -- 172.21.15.37:0/4196066322 shutdown_connections 2024-09-17T04:34:33.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.603+0000 7faf0b952640 1 -- 172.21.15.37:0/4196066322 wait complete. 2024-09-17T04:34:33.614 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:34:33.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.731+0000 7fa1d865c640 1 -- 172.21.15.37:0/1507213621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1d00b3430 msgr2=0x7fa1d00d28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:33.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.732+0000 7fa1d865c640 1 --2- 172.21.15.37:0/1507213621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1d00b3430 0x7fa1d00d28f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8005fd0 tx=0x7fa1c802aa90 comp rx=0 tx=0).stop 2024-09-17T04:34:33.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.732+0000 7fa1d865c640 1 -- 172.21.15.37:0/1507213621 shutdown_connections 2024-09-17T04:34:33.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.732+0000 7fa1d865c640 1 --2- 172.21.15.37:0/1507213621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1d00b3430 0x7fa1d00d28f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:33.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.732+0000 7fa1d865c640 1 --2- 172.21.15.37:0/1507213621 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa1d00d1610 0x7fa1d00d23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:33.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.732+0000 7fa1d865c640 1 -- 172.21.15.37:0/1507213621 >> 172.21.15.37:0/1507213621 conn(0x7fa1d008bef0 msgr2=0x7fa1d00d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:33.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.732+0000 7fa1d865c640 1 -- 172.21.15.37:0/1507213621 shutdown_connections 2024-09-17T04:34:33.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.732+0000 7fa1d865c640 1 -- 172.21.15.37:0/1507213621 wait complete. 2024-09-17T04:34:33.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.733+0000 7fa1d865c640 1 Processor -- start 2024-09-17T04:34:33.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.734+0000 7fa1d865c640 1 -- start start 2024-09-17T04:34:33.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.735+0000 7fa1d865c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa1d006f9f0 0x7fa1d00cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:33.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.735+0000 7fa1d865c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1d00b3430 0x7fa1d0156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:33.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.735+0000 7fa1d865c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1d0157440 0x7fa1d01f8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:33.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.735+0000 7fa1d865c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d00d2e30 con 0x7fa1d00b3430 2024-09-17T04:34:33.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.735+0000 7fa1d865c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d00b3b90 con 0x7fa1d0157440 2024-09-17T04:34:33.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.735+0000 7fa1d865c640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fa1d00d2fb0 con 0x7fa1d006f9f0 2024-09-17T04:34:33.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.735+0000 7fa1d5bf7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1d00b3430 0x7fa1d0156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:33.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.735+0000 7fa1d5bf7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1d00b3430 0x7fa1d0156f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48360/0 (socket says 172.21.15.37:48360) 2024-09-17T04:34:33.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.735+0000 7fa1d5bf7640 1 -- 172.21.15.37:0/4154137668 learned_addr learned my addr 172.21.15.37:0/4154137668 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:33.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.735+0000 7fa1d6bf9640 1 --2- 172.21.15.37:0/4154137668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1d0157440 0x7fa1d01f8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:33.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.735+0000 7fa1d63f8640 1 --2- 172.21.15.37:0/4154137668 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa1d006f9f0 0x7fa1d00cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:33.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.735+0000 7fa1d5bf7640 1 -- 172.21.15.37:0/4154137668 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa1d006f9f0 msgr2=0x7fa1d00cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:33.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.735+0000 7fa1d5bf7640 1 --2- 172.21.15.37:0/4154137668 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa1d006f9f0 0x7fa1d00cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:33.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.735+0000 7fa1d5bf7640 1 -- 172.21.15.37:0/4154137668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1d0157440 msgr2=0x7fa1d01f8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:33.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.735+0000 7fa1d5bf7640 1 --2- 172.21.15.37:0/4154137668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1d0157440 0x7fa1d01f8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:33.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.735+0000 7fa1d5bf7640 1 -- 172.21.15.37:0/4154137668 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d0157850 con 0x7fa1d00b3430 2024-09-17T04:34:33.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.736+0000 7fa1d5bf7640 1 --2- 172.21.15.37:0/4154137668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1d00b3430 0x7fa1d0156f00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000e9f0 tx=0x7fa1c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:33.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.736+0000 7fa1c6ffd640 1 -- 172.21.15.37:0/4154137668 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c000ccc0 con 0x7fa1d00b3430 2024-09-17T04:34:33.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.736+0000 7fa1c6ffd640 1 -- 172.21.15.37:0/4154137668 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1c0010430 con 0x7fa1d00b3430 2024-09-17T04:34:33.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.736+0000 7fa1d865c640 1 -- 172.21.15.37:0/4154137668 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d00c8160 con 0x7fa1d00b3430 2024-09-17T04:34:33.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.736+0000 7fa1c6ffd640 1 -- 172.21.15.37:0/4154137668 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0010840 con 0x7fa1d00b3430 2024-09-17T04:34:33.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.736+0000 7fa1d865c640 1 -- 172.21.15.37:0/4154137668 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d00c8420 con 0x7fa1d00b3430 2024-09-17T04:34:33.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.737+0000 7fa1d865c640 1 -- 172.21.15.37:0/4154137668 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1d00b4bd0 con 0x7fa1d00b3430 2024-09-17T04:34:33.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.738+0000 7fa1c6ffd640 1 -- 172.21.15.37:0/4154137668 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa1c00109e0 con 0x7fa1d00b3430 2024-09-17T04:34:33.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.738+0000 7fa1c6ffd640 1 --2- 172.21.15.37:0/4154137668 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa1b4078bc0 0x7fa1b407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:33.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.738+0000 7fa1c6ffd640 1 -- 172.21.15.37:0/4154137668 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7fa1c009bb50 con 0x7fa1d00b3430 2024-09-17T04:34:33.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.738+0000 7fa1d63f8640 1 --2- 172.21.15.37:0/4154137668 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa1b4078bc0 0x7fa1b407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:33.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.739+0000 7fa1d63f8640 1 --2- 172.21.15.37:0/4154137668 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa1b4078bc0 0x7fa1b407b0d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00cec30 tx=0x7fa1cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:33.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.743+0000 7fa1c6ffd640 1 -- 172.21.15.37:0/4154137668 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa1c0063cf0 con 0x7fa1d00b3430 2024-09-17T04:34:33.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.965+0000 7fa1d865c640 1 -- 172.21.15.37:0/4154137668 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1d00d3460 con 0x7fa1d00b3430 2024-09-17T04:34:33.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.965+0000 7fa1c6ffd640 1 -- 172.21.15.37:0/4154137668 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+6522 (secure 0 0 0) 0x7fa1c00635f0 con 0x7fa1d00b3430 2024-09-17T04:34:33.965 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:33.965 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"btime":"2024-09-17T04:34:28:244441+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/199192833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":199192833},{"type":"v1","addr":"172.21.15.37:6833","nonce":199192833}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4832,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/4098913371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4098913371},{"type":"v1","addr":"172.21.15.92:6836","nonce":4098913371}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/383658044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":383658044},{"type":"v1","addr":"172.21.15.92:6837","nonce":383658044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:59.490265+0000","modified":"2024-09-17T04:34:15.700096+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2},{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:34:17.928780+0000","modified":"2024-09-17T04:34:28.244429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5208":{"gid":5208,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6835/4051839052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":4051839052},{"type":"v1","addr":"172.21.15.37:6835","nonce":4051839052}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":6,"enabled":true,"fs_name":"recovery_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5208,"qdb_cluster":[5208]},"id":3}]} 2024-09-17T04:34:33.965 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-09-17T04:34:33.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.967+0000 7fa1d865c640 1 -- 172.21.15.37:0/4154137668 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa1b4078bc0 msgr2=0x7fa1b407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:33.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.967+0000 7fa1d865c640 1 --2- 172.21.15.37:0/4154137668 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa1b4078bc0 0x7fa1b407b0d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00cec30 tx=0x7fa1cc009040 comp rx=0 tx=0).stop 2024-09-17T04:34:33.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.967+0000 7fa1d865c640 1 -- 172.21.15.37:0/4154137668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1d00b3430 msgr2=0x7fa1d0156f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:33.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.967+0000 7fa1d865c640 1 --2- 172.21.15.37:0/4154137668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1d00b3430 0x7fa1d0156f00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000e9f0 tx=0x7fa1c000eeb0 comp rx=0 tx=0).stop 2024-09-17T04:34:33.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.967+0000 7fa1d865c640 1 -- 172.21.15.37:0/4154137668 shutdown_connections 2024-09-17T04:34:33.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.967+0000 7fa1d865c640 1 --2- 172.21.15.37:0/4154137668 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa1b4078bc0 0x7fa1b407b0d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:33.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.967+0000 7fa1d865c640 1 --2- 172.21.15.37:0/4154137668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1d0157440 0x7fa1d01f8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:33.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.967+0000 7fa1d865c640 1 --2- 172.21.15.37:0/4154137668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1d00b3430 0x7fa1d0156f00 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:33.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.967+0000 7fa1d865c640 1 --2- 172.21.15.37:0/4154137668 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa1d006f9f0 0x7fa1d00cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:33.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.967+0000 7fa1d865c640 1 -- 172.21.15.37:0/4154137668 >> 172.21.15.37:0/4154137668 conn(0x7fa1d008bef0 msgr2=0x7fa1d006df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:33.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.967+0000 7fa1d865c640 1 -- 172.21.15.37:0/4154137668 shutdown_connections 2024-09-17T04:34:33.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:33.967+0000 7fa1d865c640 1 -- 172.21.15.37:0/4154137668 wait complete. 2024-09-17T04:34:33.978 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:34:34.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.094+0000 7fd837c7e640 1 -- 172.21.15.37:0/3788518950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd83015b7e0 msgr2=0x7fd8300cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:34.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.094+0000 7fd837c7e640 1 --2- 172.21.15.37:0/3788518950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd83015b7e0 0x7fd8300cb790 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd820005fd0 tx=0x7fd82002a540 comp rx=0 tx=0).stop 2024-09-17T04:34:34.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.095+0000 7fd837c7e640 1 -- 172.21.15.37:0/3788518950 shutdown_connections 2024-09-17T04:34:34.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.095+0000 7fd837c7e640 1 --2- 172.21.15.37:0/3788518950 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd830070f60 0x7fd830071370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:34.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.095+0000 7fd837c7e640 1 --2- 172.21.15.37:0/3788518950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd83015b7e0 0x7fd8300cb790 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:34.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.095+0000 7fd837c7e640 1 -- 172.21.15.37:0/3788518950 >> 172.21.15.37:0/3788518950 conn(0x7fd8300d0a30 msgr2=0x7fd8300d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:34.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.095+0000 7fd837c7e640 1 -- 172.21.15.37:0/3788518950 shutdown_connections 2024-09-17T04:34:34.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.095+0000 7fd837c7e640 1 -- 172.21.15.37:0/3788518950 wait complete. 2024-09-17T04:34:34.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.096+0000 7fd837c7e640 1 Processor -- start 2024-09-17T04:34:34.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.097+0000 7fd837c7e640 1 -- start start 2024-09-17T04:34:34.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.098+0000 7fd837c7e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd830070f60 0x7fd8300b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:34.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.098+0000 7fd837c7e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8300c40c0 0x7fd8301e4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:34.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.098+0000 7fd837c7e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd83015b7e0 0x7fd8301e5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:34.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.098+0000 7fd837c7e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd8300718b0 con 0x7fd830070f60 2024-09-17T04:34:34.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.098+0000 7fd837c7e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd8300cbee0 con 0x7fd8300c40c0 2024-09-17T04:34:34.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.098+0000 7fd837c7e640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fd830071a30 con 0x7fd83015b7e0 2024-09-17T04:34:34.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.098+0000 7fd835a1a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd830070f60 0x7fd8300b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:34.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.098+0000 7fd835a1a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd830070f60 0x7fd8300b1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48388/0 (socket says 172.21.15.37:48388) 2024-09-17T04:34:34.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.098+0000 7fd835a1a640 1 -- 172.21.15.37:0/1170551321 learned_addr learned my addr 172.21.15.37:0/1170551321 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:34.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.098+0000 7fd835219640 1 --2- 172.21.15.37:0/1170551321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8300c40c0 0x7fd8301e4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:34.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.098+0000 7fd83621b640 1 --2- 172.21.15.37:0/1170551321 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd83015b7e0 0x7fd8301e5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:34.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.098+0000 7fd835a1a640 1 -- 172.21.15.37:0/1170551321 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd83015b7e0 msgr2=0x7fd8301e5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:34.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.098+0000 7fd835a1a640 1 --2- 172.21.15.37:0/1170551321 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd83015b7e0 0x7fd8301e5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:34.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.098+0000 7fd835a1a640 1 -- 172.21.15.37:0/1170551321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8300c40c0 msgr2=0x7fd8301e4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:34.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.098+0000 7fd835a1a640 1 --2- 172.21.15.37:0/1170551321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8300c40c0 0x7fd8301e4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:34.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.098+0000 7fd835a1a640 1 -- 172.21.15.37:0/1170551321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8301f9ea0 con 0x7fd830070f60 2024-09-17T04:34:34.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.098+0000 7fd835219640 1 --2- 172.21.15.37:0/1170551321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8300c40c0 0x7fd8301e4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:34.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.098+0000 7fd835a1a640 1 --2- 172.21.15.37:0/1170551321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd830070f60 0x7fd8300b1170 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd82c00c3f0 tx=0x7fd82c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:34.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.099+0000 7fd8267fc640 1 -- 172.21.15.37:0/1170551321 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c017070 con 0x7fd830070f60 2024-09-17T04:34:34.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.099+0000 7fd8267fc640 1 -- 172.21.15.37:0/1170551321 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd82c0035f0 con 0x7fd830070f60 2024-09-17T04:34:34.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.099+0000 7fd837c7e640 1 -- 172.21.15.37:0/1170551321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8301e6150 con 0x7fd830070f60 2024-09-17T04:34:34.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.099+0000 7fd8267fc640 1 -- 172.21.15.37:0/1170551321 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c003a40 con 0x7fd830070f60 2024-09-17T04:34:34.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.099+0000 7fd837c7e640 1 -- 172.21.15.37:0/1170551321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8301e6a40 con 0x7fd830070f60 2024-09-17T04:34:34.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.100+0000 7fd837c7e640 1 -- 172.21.15.37:0/1170551321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8301e6670 con 0x7fd830070f60 2024-09-17T04:34:34.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.101+0000 7fd8267fc640 1 -- 172.21.15.37:0/1170551321 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd82c01a070 con 0x7fd830070f60 2024-09-17T04:34:34.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.101+0000 7fd8267fc640 1 --2- 172.21.15.37:0/1170551321 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd814078b90 0x7fd81407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:34.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.101+0000 7fd8267fc640 1 -- 172.21.15.37:0/1170551321 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7fd82c09a640 con 0x7fd830070f60 2024-09-17T04:34:34.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.102+0000 7fd835219640 1 --2- 172.21.15.37:0/1170551321 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd814078b90 0x7fd81407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:34.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.102+0000 7fd835219640 1 --2- 172.21.15.37:0/1170551321 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd814078b90 0x7fd81407b0a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd820033000 tx=0x7fd820004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:34.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.105+0000 7fd8267fc640 1 -- 172.21.15.37:0/1170551321 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd82c021020 con 0x7fd830070f60 2024-09-17T04:34:34.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.294+0000 7fd837c7e640 1 -- 172.21.15.37:0/1170551321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd830071f70 con 0x7fd830070f60 2024-09-17T04:34:34.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.296+0000 7fd8267fc640 1 -- 172.21.15.37:0/1170551321 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+18704 (secure 0 0 0) 0x7fd8300704e0 con 0x7fd830070f60 2024-09-17T04:34:34.296 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:34.296 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":41,"fsid":"a908a603-430c-4209-8294-0c3460b35f42","created":"2024-09-17T04:32:54.139221+0000","modified":"2024-09-17T04:34:19.179872+0000","last_up_change":"2024-09-17T04:33:06.922210+0000","last_in_change":"2024-09-17T04:32:57.754118+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:33:08.461907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:33:57.558905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:58.446650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"recovery_fs_metadata","create_time":"2024-09-17T04:34:16.436885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ea4b8d1-0d22-4dd9-b913-9df585ef24ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6809","nonce":2283587593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6811","nonce":2283587593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6815","nonce":2283587593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6813","nonce":2283587593}]},"public_addr":"172.21.15.37:6809/2283587593","cluster_addr":"172.21.15.37:6811/2283587593","heartbeat_back_addr":"172.21.15.37:6815/2283587593","heartbeat_front_addr":"172.21.15.37:6813/2283587593","state":["exists","up"]},{"osd":1,"uuid":"ac745a33-e8ad-4083-9c18-71bf81982b76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6825","nonce":2657955935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6827","nonce":2657955935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6831","nonce":2657955935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6829","nonce":2657955935}]},"public_addr":"172.21.15.37:6825/2657955935","cluster_addr":"172.21.15.37:6827/2657955935","heartbeat_back_addr":"172.21.15.37:6831/2657955935","heartbeat_front_addr":"172.21.15.37:6829/2657955935","state":["exists","up"]},{"osd":2,"uuid":"3c63f3d8-132a-4735-b0bc-d64313a46503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6817","nonce":579720176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6819","nonce":579720176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6823","nonce":579720176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6821","nonce":579720176}]},"public_addr":"172.21.15.37:6817/579720176","cluster_addr":"172.21.15.37:6819/579720176","heartbeat_back_addr":"172.21.15.37:6823/579720176","heartbeat_front_addr":"172.21.15.37:6821/579720176","state":["exists","up"]},{"osd":3,"uuid":"4f8c6ca2-ea1d-4610-9127-52c4dd8bc8e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6801","nonce":809975826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6803","nonce":809975826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6807","nonce":809975826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6805","nonce":809975826}]},"public_addr":"172.21.15.37:6801/809975826","cluster_addr":"172.21.15.37:6803/809975826","heartbeat_back_addr":"172.21.15.37:6807/809975826","heartbeat_front_addr":"172.21.15.37:6805/809975826","state":["exists","up"]},{"osd":4,"uuid":"63d0c4ab-f115-47b4-9611-12a10153473a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6819","nonce":2509785752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6821","nonce":2509785752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6825","nonce":2509785752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6823","nonce":2509785752}]},"public_addr":"172.21.15.92:6819/2509785752","cluster_addr":"172.21.15.92:6821/2509785752","heartbeat_back_addr":"172.21.15.92:6825/2509785752","heartbeat_front_addr":"172.21.15.92:6823/2509785752","state":["exists","up"]},{"osd":5,"uuid":"4c56c04d-0c07-4f76-9d75-6335121310ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6827","nonce":248263832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6829","nonce":248263832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6833","nonce":248263832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6831","nonce":248263832}]},"public_addr":"172.21.15.92:6827/248263832","cluster_addr":"172.21.15.92:6829/248263832","heartbeat_back_addr":"172.21.15.92:6833/248263832","heartbeat_front_addr":"172.21.15.92:6831/248263832","state":["exists","up"]},{"osd":6,"uuid":"b30eae7c-096a-4d24-ac51-96b8b13ecaee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6811","nonce":4078365985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6813","nonce":4078365985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6817","nonce":4078365985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6815","nonce":4078365985}]},"public_addr":"172.21.15.92:6811/4078365985","cluster_addr":"172.21.15.92:6813/4078365985","heartbeat_back_addr":"172.21.15.92:6817/4078365985","heartbeat_front_addr":"172.21.15.92:6815/4078365985","state":["exists","up"]},{"osd":7,"uuid":"62f12540-d374-4a4c-9ff7-63b7304fe77f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6803","nonce":3444787223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6805","nonce":3444787223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6809","nonce":3444787223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6807","nonce":3444787223}]},"public_addr":"172.21.15.92:6803/3444787223","cluster_addr":"172.21.15.92:6805/3444787223","heartbeat_back_addr":"172.21.15.92:6809/3444787223","heartbeat_front_addr":"172.21.15.92:6807/3444787223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.006420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.111135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.103325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.072656+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.200989+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.060455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.774773+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.865778+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/1358600605":"2024-09-18T04:34:15.699884+0000","172.21.15.37:6833/1358600605":"2024-09-18T04:34:15.699884+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:34:34.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.298+0000 7fd837c7e640 1 -- 172.21.15.37:0/1170551321 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd814078b90 msgr2=0x7fd81407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:34.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.298+0000 7fd837c7e640 1 --2- 172.21.15.37:0/1170551321 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd814078b90 0x7fd81407b0a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd820033000 tx=0x7fd820004730 comp rx=0 tx=0).stop 2024-09-17T04:34:34.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.298+0000 7fd837c7e640 1 -- 172.21.15.37:0/1170551321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd830070f60 msgr2=0x7fd8300b1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:34.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.298+0000 7fd837c7e640 1 --2- 172.21.15.37:0/1170551321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd830070f60 0x7fd8300b1170 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd82c00c3f0 tx=0x7fd82c00c8b0 comp rx=0 tx=0).stop 2024-09-17T04:34:34.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.298+0000 7fd837c7e640 1 -- 172.21.15.37:0/1170551321 shutdown_connections 2024-09-17T04:34:34.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.298+0000 7fd837c7e640 1 --2- 172.21.15.37:0/1170551321 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fd814078b90 0x7fd81407b0a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:34.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.298+0000 7fd837c7e640 1 --2- 172.21.15.37:0/1170551321 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd83015b7e0 0x7fd8301e5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:34.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.298+0000 7fd837c7e640 1 --2- 172.21.15.37:0/1170551321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8300c40c0 0x7fd8301e4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:34.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.298+0000 7fd837c7e640 1 --2- 172.21.15.37:0/1170551321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd830070f60 0x7fd8300b1170 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:34.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.298+0000 7fd837c7e640 1 -- 172.21.15.37:0/1170551321 >> 172.21.15.37:0/1170551321 conn(0x7fd8300d0a30 msgr2=0x7fd8300ce790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:34.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.298+0000 7fd837c7e640 1 -- 172.21.15.37:0/1170551321 shutdown_connections 2024-09-17T04:34:34.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.298+0000 7fd837c7e640 1 -- 172.21.15.37:0/1170551321 wait complete. 2024-09-17T04:34:34.310 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T04:34:34.310 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:34:34.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.429+0000 7f649b45c640 1 -- 172.21.15.37:0/1252374223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64940d1610 msgr2=0x7f64940d23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:34.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.429+0000 7f649b45c640 1 --2- 172.21.15.37:0/1252374223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64940d1610 0x7f64940d23b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6484009820 tx=0x7f648402a5a0 comp rx=0 tx=0).stop 2024-09-17T04:34:34.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.429+0000 7f649b45c640 1 -- 172.21.15.37:0/1252374223 shutdown_connections 2024-09-17T04:34:34.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.429+0000 7f649b45c640 1 --2- 172.21.15.37:0/1252374223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64940d1610 0x7f64940d23b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:34.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.429+0000 7f649b45c640 1 -- 172.21.15.37:0/1252374223 >> 172.21.15.37:0/1252374223 conn(0x7f649408bef0 msgr2=0x7f64940d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:34.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.429+0000 7f649b45c640 1 -- 172.21.15.37:0/1252374223 shutdown_connections 2024-09-17T04:34:34.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.429+0000 7f649b45c640 1 -- 172.21.15.37:0/1252374223 wait complete. 2024-09-17T04:34:34.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.431+0000 7f649b45c640 1 Processor -- start 2024-09-17T04:34:34.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.431+0000 7f649b45c640 1 -- start start 2024-09-17T04:34:34.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.432+0000 7f649b45c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f649406f9f0 0x7f64940cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:34.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.432+0000 7f649b45c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f64940b3430 0x7f64941f8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:34.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.432+0000 7f649b45c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64941f9380 0x7f64941f9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:34.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.432+0000 7f649b45c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f64940d3000 con 0x7f64941f9380 2024-09-17T04:34:34.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.432+0000 7f649b45c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f64940b3b90 con 0x7f649406f9f0 2024-09-17T04:34:34.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.432+0000 7f649b45c640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f64940d3180 con 0x7f64940b3430 2024-09-17T04:34:34.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.432+0000 7f64999f9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64941f9380 0x7f64941f9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:34.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.432+0000 7f64999f9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64941f9380 0x7f64941f9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48396/0 (socket says 172.21.15.37:48396) 2024-09-17T04:34:34.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.432+0000 7f64999f9640 1 -- 172.21.15.37:0/3174456484 learned_addr learned my addr 172.21.15.37:0/3174456484 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:34.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.432+0000 7f64991f8640 1 --2- 172.21.15.37:0/3174456484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f649406f9f0 0x7f64940cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:34.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.432+0000 7f64989f7640 1 --2- 172.21.15.37:0/3174456484 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f64940b3430 0x7f64941f8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:34.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.433+0000 7f64999f9640 1 -- 172.21.15.37:0/3174456484 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f64940b3430 msgr2=0x7f64941f8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:34.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.433+0000 7f64999f9640 1 --2- 172.21.15.37:0/3174456484 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f64940b3430 0x7f64941f8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:34.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.433+0000 7f64999f9640 1 -- 172.21.15.37:0/3174456484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f649406f9f0 msgr2=0x7f64940cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:34.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.433+0000 7f64999f9640 1 --2- 172.21.15.37:0/3174456484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f649406f9f0 0x7f64940cdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:34.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.433+0000 7f64999f9640 1 -- 172.21.15.37:0/3174456484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64940b5af0 con 0x7f64941f9380 2024-09-17T04:34:34.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.433+0000 7f64999f9640 1 --2- 172.21.15.37:0/3174456484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64941f9380 0x7f64941f9790 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f648c00c700 tx=0x7f648c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:34.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.433+0000 7f6489ffb640 1 -- 172.21.15.37:0/3174456484 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c003b70 con 0x7f64941f9380 2024-09-17T04:34:34.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.433+0000 7f6489ffb640 1 -- 172.21.15.37:0/3174456484 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f648c003d10 con 0x7f64941f9380 2024-09-17T04:34:34.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.433+0000 7f649b45c640 1 -- 172.21.15.37:0/3174456484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64940b5f90 con 0x7f64941f9380 2024-09-17T04:34:34.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.433+0000 7f6489ffb640 1 -- 172.21.15.37:0/3174456484 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c00aab0 con 0x7f64941f9380 2024-09-17T04:34:34.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.433+0000 7f649b45c640 1 -- 172.21.15.37:0/3174456484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64940b6550 con 0x7f64941f9380 2024-09-17T04:34:34.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.435+0000 7f649b45c640 1 -- 172.21.15.37:0/3174456484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64940d16c0 con 0x7f64941f9380 2024-09-17T04:34:34.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.435+0000 7f6489ffb640 1 -- 172.21.15.37:0/3174456484 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f648c00ac50 con 0x7f64941f9380 2024-09-17T04:34:34.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.435+0000 7f6489ffb640 1 --2- 172.21.15.37:0/3174456484 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6478078b90 0x7f647807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:34.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.436+0000 7f6489ffb640 1 -- 172.21.15.37:0/3174456484 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7f648c09ab30 con 0x7f64941f9380 2024-09-17T04:34:34.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.436+0000 7f64991f8640 1 --2- 172.21.15.37:0/3174456484 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6478078b90 0x7f647807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:34.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.436+0000 7f64991f8640 1 --2- 172.21.15.37:0/3174456484 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6478078b90 0x7f647807b0a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f64900046b0 tx=0x7f6490007cc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:34.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.440+0000 7f6489ffb640 1 -- 172.21.15.37:0/3174456484 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f648c062820 con 0x7f64941f9380 2024-09-17T04:34:34.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.664+0000 7f649b45c640 1 -- 172.21.15.37:0/3174456484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64940b5d20 con 0x7f64941f9380 2024-09-17T04:34:34.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.664+0000 7f6489ffb640 1 -- 172.21.15.37:0/3174456484 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+6522 (secure 0 0 0) 0x7f648c067250 con 0x7f64941f9380 2024-09-17T04:34:34.664 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:34.665 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"btime":"2024-09-17T04:34:28:244441+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/199192833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":199192833},{"type":"v1","addr":"172.21.15.37:6833","nonce":199192833}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4832,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/4098913371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4098913371},{"type":"v1","addr":"172.21.15.92:6836","nonce":4098913371}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/383658044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":383658044},{"type":"v1","addr":"172.21.15.92:6837","nonce":383658044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:59.490265+0000","modified":"2024-09-17T04:34:15.700096+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2},{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:34:17.928780+0000","modified":"2024-09-17T04:34:28.244429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5208":{"gid":5208,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6835/4051839052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":4051839052},{"type":"v1","addr":"172.21.15.37:6835","nonce":4051839052}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":6,"enabled":true,"fs_name":"recovery_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5208,"qdb_cluster":[5208]},"id":3}]} 2024-09-17T04:34:34.665 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-09-17T04:34:34.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.666+0000 7f649b45c640 1 -- 172.21.15.37:0/3174456484 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6478078b90 msgr2=0x7f647807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:34.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.666+0000 7f649b45c640 1 --2- 172.21.15.37:0/3174456484 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6478078b90 0x7f647807b0a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f64900046b0 tx=0x7f6490007cc0 comp rx=0 tx=0).stop 2024-09-17T04:34:34.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.666+0000 7f649b45c640 1 -- 172.21.15.37:0/3174456484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64941f9380 msgr2=0x7f64941f9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:34.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.666+0000 7f649b45c640 1 --2- 172.21.15.37:0/3174456484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64941f9380 0x7f64941f9790 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f648c00c700 tx=0x7f648c00cbc0 comp rx=0 tx=0).stop 2024-09-17T04:34:34.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.666+0000 7f649b45c640 1 -- 172.21.15.37:0/3174456484 shutdown_connections 2024-09-17T04:34:34.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.666+0000 7f649b45c640 1 --2- 172.21.15.37:0/3174456484 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6478078b90 0x7f647807b0a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:34.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.666+0000 7f649b45c640 1 --2- 172.21.15.37:0/3174456484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64941f9380 0x7f64941f9790 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:34.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.666+0000 7f649b45c640 1 --2- 172.21.15.37:0/3174456484 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f64940b3430 0x7f64941f8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:34.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.666+0000 7f649b45c640 1 --2- 172.21.15.37:0/3174456484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f649406f9f0 0x7f64940cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:34.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.666+0000 7f649b45c640 1 -- 172.21.15.37:0/3174456484 >> 172.21.15.37:0/3174456484 conn(0x7f649408bef0 msgr2=0x7f64940b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:34.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.667+0000 7f649b45c640 1 -- 172.21.15.37:0/3174456484 shutdown_connections 2024-09-17T04:34:34.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.667+0000 7f649b45c640 1 -- 172.21.15.37:0/3174456484 wait complete. 2024-09-17T04:34:34.677 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:34:34.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.794+0000 7fbc372bd640 1 -- 172.21.15.37:0/689790038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc300d1440 msgr2=0x7fbc30162d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:34.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.794+0000 7fbc372bd640 1 --2- 172.21.15.37:0/689790038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc300d1440 0x7fbc30162d50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbc3008c9f0 tx=0x7fbc2802a5f0 comp rx=0 tx=0).stop 2024-09-17T04:34:34.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.794+0000 7fbc372bd640 1 -- 172.21.15.37:0/689790038 shutdown_connections 2024-09-17T04:34:34.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.794+0000 7fbc372bd640 1 --2- 172.21.15.37:0/689790038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc300d1440 0x7fbc30162d50 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:34.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.794+0000 7fbc372bd640 1 --2- 172.21.15.37:0/689790038 >> v2:172.21.15.92:3300/0 conn(0x7fbc3006f9f0 0x7fbc300d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:34.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.794+0000 7fbc372bd640 1 -- 172.21.15.37:0/689790038 >> 172.21.15.37:0/689790038 conn(0x7fbc3008bef0 msgr2=0x7fbc300d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:34.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.794+0000 7fbc372bd640 1 -- 172.21.15.37:0/689790038 shutdown_connections 2024-09-17T04:34:34.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.794+0000 7fbc372bd640 1 -- 172.21.15.37:0/689790038 wait complete. 2024-09-17T04:34:34.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.796+0000 7fbc372bd640 1 Processor -- start 2024-09-17T04:34:34.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.796+0000 7fbc372bd640 1 -- start start 2024-09-17T04:34:34.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.797+0000 7fbc372bd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbc3006f9f0 0x7fbc301f9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:34.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.797+0000 7fbc372bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc300b3430 0x7fbc301f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:34.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.797+0000 7fbc372bd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc300d1440 0x7fbc3015e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:34.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.797+0000 7fbc372bd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbc300b3b00 con 0x7fbc300b3430 2024-09-17T04:34:34.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.797+0000 7fbc372bd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbc300b3980 con 0x7fbc300d1440 2024-09-17T04:34:34.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.797+0000 7fbc372bd640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fbc30163290 con 0x7fbc3006f9f0 2024-09-17T04:34:34.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.797+0000 7fbc34858640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc300b3430 0x7fbc301f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:34.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.798+0000 7fbc34858640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc300b3430 0x7fbc301f66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48410/0 (socket says 172.21.15.37:48410) 2024-09-17T04:34:34.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.798+0000 7fbc34858640 1 -- 172.21.15.37:0/2382560771 learned_addr learned my addr 172.21.15.37:0/2382560771 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:34.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.798+0000 7fbc3585a640 1 --2- 172.21.15.37:0/2382560771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc300d1440 0x7fbc3015e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:34.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.798+0000 7fbc35059640 1 --2- 172.21.15.37:0/2382560771 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbc3006f9f0 0x7fbc301f9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:34.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.798+0000 7fbc34858640 1 -- 172.21.15.37:0/2382560771 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbc3006f9f0 msgr2=0x7fbc301f9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:34.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.798+0000 7fbc34858640 1 --2- 172.21.15.37:0/2382560771 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbc3006f9f0 0x7fbc301f9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:34.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.798+0000 7fbc34858640 1 -- 172.21.15.37:0/2382560771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc300d1440 msgr2=0x7fbc3015e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:34.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.798+0000 7fbc34858640 1 --2- 172.21.15.37:0/2382560771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc300d1440 0x7fbc3015e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:34.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.798+0000 7fbc34858640 1 -- 172.21.15.37:0/2382560771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc301f92a0 con 0x7fbc300b3430 2024-09-17T04:34:34.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.798+0000 7fbc3585a640 1 --2- 172.21.15.37:0/2382560771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc300d1440 0x7fbc3015e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:34.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.798+0000 7fbc34858640 1 --2- 172.21.15.37:0/2382560771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc300b3430 0x7fbc301f66a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400c760 tx=0x7fbc2400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:34.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.798+0000 7fbc2dffb640 1 -- 172.21.15.37:0/2382560771 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc24003b70 con 0x7fbc300b3430 2024-09-17T04:34:34.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.798+0000 7fbc2dffb640 1 -- 172.21.15.37:0/2382560771 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc24003d10 con 0x7fbc300b3430 2024-09-17T04:34:34.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.798+0000 7fbc372bd640 1 -- 172.21.15.37:0/2382560771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc300cac20 con 0x7fbc300b3430 2024-09-17T04:34:34.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.798+0000 7fbc2dffb640 1 -- 172.21.15.37:0/2382560771 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2400aab0 con 0x7fbc300b3430 2024-09-17T04:34:34.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.798+0000 7fbc372bd640 1 -- 172.21.15.37:0/2382560771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc300cc140 con 0x7fbc300b3430 2024-09-17T04:34:34.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.799+0000 7fbc372bd640 1 -- 172.21.15.37:0/2382560771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc300d34a0 con 0x7fbc300b3430 2024-09-17T04:34:34.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.800+0000 7fbc2dffb640 1 -- 172.21.15.37:0/2382560771 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbc2400ac50 con 0x7fbc300b3430 2024-09-17T04:34:34.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.801+0000 7fbc2dffb640 1 --2- 172.21.15.37:0/2382560771 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fbc1c078b90 0x7fbc1c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:34.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.801+0000 7fbc2dffb640 1 -- 172.21.15.37:0/2382560771 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7fbc2409af00 con 0x7fbc300b3430 2024-09-17T04:34:34.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.801+0000 7fbc35059640 1 --2- 172.21.15.37:0/2382560771 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fbc1c078b90 0x7fbc1c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:34.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.802+0000 7fbc35059640 1 --2- 172.21.15.37:0/2382560771 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fbc1c078b90 0x7fbc1c07b0a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbc200058c0 tx=0x7fbc20005850 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:34.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:34.805+0000 7fbc2dffb640 1 -- 172.21.15.37:0/2382560771 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fbc300d1850 con 0x7fbc300b3430 2024-09-17T04:34:35.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.028+0000 7fbc372bd640 1 -- 172.21.15.37:0/2382560771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc3006a150 con 0x7fbc300b3430 2024-09-17T04:34:35.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.029+0000 7fbc2dffb640 1 -- 172.21.15.37:0/2382560771 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+6522 (secure 0 0 0) 0x7fbc24063000 con 0x7fbc300b3430 2024-09-17T04:34:35.029 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:35.029 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"btime":"2024-09-17T04:34:28:244441+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/199192833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":199192833},{"type":"v1","addr":"172.21.15.37:6833","nonce":199192833}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4832,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/4098913371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4098913371},{"type":"v1","addr":"172.21.15.92:6836","nonce":4098913371}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/383658044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":383658044},{"type":"v1","addr":"172.21.15.92:6837","nonce":383658044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:33:59.490265+0000","modified":"2024-09-17T04:34:15.700096+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2},{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:34:17.928780+0000","modified":"2024-09-17T04:34:28.244429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5208":{"gid":5208,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6835/4051839052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":4051839052},{"type":"v1","addr":"172.21.15.37:6835","nonce":4051839052}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":6,"enabled":true,"fs_name":"recovery_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5208,"qdb_cluster":[5208]},"id":3}]} 2024-09-17T04:34:35.029 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-09-17T04:34:35.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.031+0000 7fbc372bd640 1 -- 172.21.15.37:0/2382560771 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fbc1c078b90 msgr2=0x7fbc1c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:35.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.031+0000 7fbc372bd640 1 --2- 172.21.15.37:0/2382560771 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fbc1c078b90 0x7fbc1c07b0a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbc200058c0 tx=0x7fbc20005850 comp rx=0 tx=0).stop 2024-09-17T04:34:35.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.031+0000 7fbc372bd640 1 -- 172.21.15.37:0/2382560771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc300b3430 msgr2=0x7fbc301f66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:35.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.031+0000 7fbc372bd640 1 --2- 172.21.15.37:0/2382560771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc300b3430 0x7fbc301f66a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400c760 tx=0x7fbc2400cc20 comp rx=0 tx=0).stop 2024-09-17T04:34:35.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.031+0000 7fbc372bd640 1 -- 172.21.15.37:0/2382560771 shutdown_connections 2024-09-17T04:34:35.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.031+0000 7fbc372bd640 1 --2- 172.21.15.37:0/2382560771 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fbc1c078b90 0x7fbc1c07b0a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:35.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.031+0000 7fbc372bd640 1 --2- 172.21.15.37:0/2382560771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc300d1440 0x7fbc3015e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:35.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.031+0000 7fbc372bd640 1 --2- 172.21.15.37:0/2382560771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc300b3430 0x7fbc301f66a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:35.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.031+0000 7fbc372bd640 1 --2- 172.21.15.37:0/2382560771 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbc3006f9f0 0x7fbc301f9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:35.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.031+0000 7fbc372bd640 1 -- 172.21.15.37:0/2382560771 >> 172.21.15.37:0/2382560771 conn(0x7fbc3008bef0 msgr2=0x7fbc300b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:35.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.031+0000 7fbc372bd640 1 -- 172.21.15.37:0/2382560771 shutdown_connections 2024-09-17T04:34:35.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.031+0000 7fbc372bd640 1 -- 172.21.15.37:0/2382560771 wait complete. 2024-09-17T04:34:35.042 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:34:35.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.159+0000 7f4391162640 1 -- 172.21.15.37:0/2519090499 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f438c0d1610 msgr2=0x7f438c0d23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:35.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.159+0000 7f4391162640 1 --2- 172.21.15.37:0/2519090499 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f438c0d1610 0x7f438c0d23b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4380009850 tx=0x7f438002a5a0 comp rx=0 tx=0).stop 2024-09-17T04:34:35.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.159+0000 7f4391162640 1 -- 172.21.15.37:0/2519090499 shutdown_connections 2024-09-17T04:34:35.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.159+0000 7f4391162640 1 --2- 172.21.15.37:0/2519090499 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f438c0d1610 0x7f438c0d23b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:35.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.159+0000 7f4391162640 1 -- 172.21.15.37:0/2519090499 >> 172.21.15.37:0/2519090499 conn(0x7f438c08bef0 msgr2=0x7f438c0d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:35.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.159+0000 7f4391162640 1 -- 172.21.15.37:0/2519090499 shutdown_connections 2024-09-17T04:34:35.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.159+0000 7f4391162640 1 -- 172.21.15.37:0/2519090499 wait complete. 2024-09-17T04:34:35.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.161+0000 7f4391162640 1 Processor -- start 2024-09-17T04:34:35.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.162+0000 7f4391162640 1 -- start start 2024-09-17T04:34:35.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.163+0000 7f4391162640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f438c06f9f0 0x7f438c0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:35.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.163+0000 7f4391162640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f438c0b3430 0x7f438c0b62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:35.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.163+0000 7f4391162640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f438c0b6800 0x7f438c1f9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:35.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.163+0000 7f4391162640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f438c0d3000 con 0x7f438c06f9f0 2024-09-17T04:34:35.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.163+0000 7f4391162640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f438c0b3b90 con 0x7f438c0b3430 2024-09-17T04:34:35.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.163+0000 7f4391162640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f438c0d3180 con 0x7f438c0b6800 2024-09-17T04:34:35.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.163+0000 7f438ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f438c06f9f0 0x7f438c0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:35.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.163+0000 7f438ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f438c06f9f0 0x7f438c0cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48414/0 (socket says 172.21.15.37:48414) 2024-09-17T04:34:35.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.163+0000 7f438ad9d640 1 -- 172.21.15.37:0/1768846825 learned_addr learned my addr 172.21.15.37:0/1768846825 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:35.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.163+0000 7f438a59c640 1 --2- 172.21.15.37:0/1768846825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f438c0b3430 0x7f438c0b62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:35.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.163+0000 7f438b59e640 1 --2- 172.21.15.37:0/1768846825 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f438c0b6800 0x7f438c1f9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:35.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.163+0000 7f438ad9d640 1 -- 172.21.15.37:0/1768846825 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f438c0b6800 msgr2=0x7f438c1f9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:35.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.163+0000 7f438ad9d640 1 --2- 172.21.15.37:0/1768846825 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f438c0b6800 0x7f438c1f9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:35.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.163+0000 7f438ad9d640 1 -- 172.21.15.37:0/1768846825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f438c0b3430 msgr2=0x7f438c0b62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:35.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.163+0000 7f438ad9d640 1 --2- 172.21.15.37:0/1768846825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f438c0b3430 0x7f438c0b62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:35.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.163+0000 7f438ad9d640 1 -- 172.21.15.37:0/1768846825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f438c0c9de0 con 0x7f438c06f9f0 2024-09-17T04:34:35.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.163+0000 7f438b59e640 1 --2- 172.21.15.37:0/1768846825 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f438c0b6800 0x7f438c1f9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:35.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.163+0000 7f438ad9d640 1 --2- 172.21.15.37:0/1768846825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f438c06f9f0 0x7f438c0cdfb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f437c00c430 tx=0x7f437c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:35.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.164+0000 7f43737fe640 1 -- 172.21.15.37:0/1768846825 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c017070 con 0x7f438c06f9f0 2024-09-17T04:34:35.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.164+0000 7f43737fe640 1 -- 172.21.15.37:0/1768846825 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f437c0053e0 con 0x7f438c06f9f0 2024-09-17T04:34:35.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.164+0000 7f4391162640 1 -- 172.21.15.37:0/1768846825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f438c0ca0c0 con 0x7f438c06f9f0 2024-09-17T04:34:35.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.164+0000 7f43737fe640 1 -- 172.21.15.37:0/1768846825 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c003850 con 0x7f438c06f9f0 2024-09-17T04:34:35.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.164+0000 7f4391162640 1 -- 172.21.15.37:0/1768846825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f438c0cb930 con 0x7f438c06f9f0 2024-09-17T04:34:35.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.165+0000 7f4391162640 1 -- 172.21.15.37:0/1768846825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f438c0d0f00 con 0x7f438c06f9f0 2024-09-17T04:34:35.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.166+0000 7f43737fe640 1 -- 172.21.15.37:0/1768846825 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f437c01a070 con 0x7f438c06f9f0 2024-09-17T04:34:35.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.166+0000 7f43737fe640 1 --2- 172.21.15.37:0/1768846825 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4378078b90 0x7f437807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:35.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.166+0000 7f43737fe640 1 -- 172.21.15.37:0/1768846825 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7f437c09a230 con 0x7f438c06f9f0 2024-09-17T04:34:35.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.166+0000 7f438a59c640 1 --2- 172.21.15.37:0/1768846825 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4378078b90 0x7f437807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:35.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.167+0000 7f438a59c640 1 --2- 172.21.15.37:0/1768846825 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4378078b90 0x7f437807b0a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4380034000 tx=0x7f4380004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:35.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.171+0000 7f43737fe640 1 -- 172.21.15.37:0/1768846825 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f437c061e90 con 0x7f438c06f9f0 2024-09-17T04:34:35.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.357+0000 7f4391162640 1 -- 172.21.15.37:0/1768846825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f438c06a150 con 0x7f438c06f9f0 2024-09-17T04:34:35.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.358+0000 7f43737fe640 1 -- 172.21.15.37:0/1768846825 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+18704 (secure 0 0 0) 0x7f437c0668c0 con 0x7f438c06f9f0 2024-09-17T04:34:35.358 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:35.359 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":41,"fsid":"a908a603-430c-4209-8294-0c3460b35f42","created":"2024-09-17T04:32:54.139221+0000","modified":"2024-09-17T04:34:19.179872+0000","last_up_change":"2024-09-17T04:33:06.922210+0000","last_in_change":"2024-09-17T04:32:57.754118+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:33:08.461907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T04:33:57.558905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:58.446650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"recovery_fs_metadata","create_time":"2024-09-17T04:34:16.436885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ea4b8d1-0d22-4dd9-b913-9df585ef24ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6809","nonce":2283587593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6811","nonce":2283587593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6815","nonce":2283587593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6813","nonce":2283587593}]},"public_addr":"172.21.15.37:6809/2283587593","cluster_addr":"172.21.15.37:6811/2283587593","heartbeat_back_addr":"172.21.15.37:6815/2283587593","heartbeat_front_addr":"172.21.15.37:6813/2283587593","state":["exists","up"]},{"osd":1,"uuid":"ac745a33-e8ad-4083-9c18-71bf81982b76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6825","nonce":2657955935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6827","nonce":2657955935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6831","nonce":2657955935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6829","nonce":2657955935}]},"public_addr":"172.21.15.37:6825/2657955935","cluster_addr":"172.21.15.37:6827/2657955935","heartbeat_back_addr":"172.21.15.37:6831/2657955935","heartbeat_front_addr":"172.21.15.37:6829/2657955935","state":["exists","up"]},{"osd":2,"uuid":"3c63f3d8-132a-4735-b0bc-d64313a46503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6817","nonce":579720176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6819","nonce":579720176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6823","nonce":579720176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6821","nonce":579720176}]},"public_addr":"172.21.15.37:6817/579720176","cluster_addr":"172.21.15.37:6819/579720176","heartbeat_back_addr":"172.21.15.37:6823/579720176","heartbeat_front_addr":"172.21.15.37:6821/579720176","state":["exists","up"]},{"osd":3,"uuid":"4f8c6ca2-ea1d-4610-9127-52c4dd8bc8e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6801","nonce":809975826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6803","nonce":809975826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6807","nonce":809975826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6805","nonce":809975826}]},"public_addr":"172.21.15.37:6801/809975826","cluster_addr":"172.21.15.37:6803/809975826","heartbeat_back_addr":"172.21.15.37:6807/809975826","heartbeat_front_addr":"172.21.15.37:6805/809975826","state":["exists","up"]},{"osd":4,"uuid":"63d0c4ab-f115-47b4-9611-12a10153473a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6819","nonce":2509785752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6821","nonce":2509785752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6825","nonce":2509785752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6823","nonce":2509785752}]},"public_addr":"172.21.15.92:6819/2509785752","cluster_addr":"172.21.15.92:6821/2509785752","heartbeat_back_addr":"172.21.15.92:6825/2509785752","heartbeat_front_addr":"172.21.15.92:6823/2509785752","state":["exists","up"]},{"osd":5,"uuid":"4c56c04d-0c07-4f76-9d75-6335121310ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6827","nonce":248263832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6829","nonce":248263832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6833","nonce":248263832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6831","nonce":248263832}]},"public_addr":"172.21.15.92:6827/248263832","cluster_addr":"172.21.15.92:6829/248263832","heartbeat_back_addr":"172.21.15.92:6833/248263832","heartbeat_front_addr":"172.21.15.92:6831/248263832","state":["exists","up"]},{"osd":6,"uuid":"b30eae7c-096a-4d24-ac51-96b8b13ecaee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6811","nonce":4078365985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6813","nonce":4078365985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6817","nonce":4078365985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6815","nonce":4078365985}]},"public_addr":"172.21.15.92:6811/4078365985","cluster_addr":"172.21.15.92:6813/4078365985","heartbeat_back_addr":"172.21.15.92:6817/4078365985","heartbeat_front_addr":"172.21.15.92:6815/4078365985","state":["exists","up"]},{"osd":7,"uuid":"62f12540-d374-4a4c-9ff7-63b7304fe77f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6803","nonce":3444787223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6805","nonce":3444787223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6809","nonce":3444787223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6807","nonce":3444787223}]},"public_addr":"172.21.15.92:6803/3444787223","cluster_addr":"172.21.15.92:6805/3444787223","heartbeat_back_addr":"172.21.15.92:6809/3444787223","heartbeat_front_addr":"172.21.15.92:6807/3444787223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.006420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.111135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.103325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.072656+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.200989+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.060455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.774773+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.865778+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/1358600605":"2024-09-18T04:34:15.699884+0000","172.21.15.37:6833/1358600605":"2024-09-18T04:34:15.699884+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:34:35.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.361+0000 7f4391162640 1 -- 172.21.15.37:0/1768846825 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4378078b90 msgr2=0x7f437807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:35.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.361+0000 7f4391162640 1 --2- 172.21.15.37:0/1768846825 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4378078b90 0x7f437807b0a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4380034000 tx=0x7f4380004730 comp rx=0 tx=0).stop 2024-09-17T04:34:35.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.361+0000 7f4391162640 1 -- 172.21.15.37:0/1768846825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f438c06f9f0 msgr2=0x7f438c0cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:35.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.361+0000 7f4391162640 1 --2- 172.21.15.37:0/1768846825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f438c06f9f0 0x7f438c0cdfb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f437c00c430 tx=0x7f437c00c8f0 comp rx=0 tx=0).stop 2024-09-17T04:34:35.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.361+0000 7f4391162640 1 -- 172.21.15.37:0/1768846825 shutdown_connections 2024-09-17T04:34:35.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.361+0000 7f4391162640 1 --2- 172.21.15.37:0/1768846825 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4378078b90 0x7f437807b0a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:35.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.361+0000 7f4391162640 1 --2- 172.21.15.37:0/1768846825 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f438c0b6800 0x7f438c1f9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:35.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.361+0000 7f4391162640 1 --2- 172.21.15.37:0/1768846825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f438c0b3430 0x7f438c0b62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:35.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.361+0000 7f4391162640 1 --2- 172.21.15.37:0/1768846825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f438c06f9f0 0x7f438c0cdfb0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:35.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.361+0000 7f4391162640 1 -- 172.21.15.37:0/1768846825 >> 172.21.15.37:0/1768846825 conn(0x7f438c08bef0 msgr2=0x7f438c0b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:35.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.361+0000 7f4391162640 1 -- 172.21.15.37:0/1768846825 shutdown_connections 2024-09-17T04:34:35.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.361+0000 7f4391162640 1 -- 172.21.15.37:0/1768846825 wait complete. 2024-09-17T04:34:35.373 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T04:34:35.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.490+0000 7fcd61084640 1 -- 172.21.15.37:0/3253646735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5c0b15b0 msgr2=0x7fcd5c0b4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:35.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.490+0000 7fcd61084640 1 --2- 172.21.15.37:0/3253646735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5c0b15b0 0x7fcd5c0b4240 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fcd44005fd0 tx=0x7fcd4402a5d0 comp rx=0 tx=0).stop 2024-09-17T04:34:35.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.490+0000 7fcd61084640 1 -- 172.21.15.37:0/3253646735 shutdown_connections 2024-09-17T04:34:35.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.490+0000 7fcd61084640 1 --2- 172.21.15.37:0/3253646735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5c0b15b0 0x7fcd5c0b4240 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:35.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.490+0000 7fcd61084640 1 --2- 172.21.15.37:0/3253646735 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcd5c06d7d0 0x7fcd5c160d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:35.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.491+0000 7fcd61084640 1 -- 172.21.15.37:0/3253646735 >> 172.21.15.37:0/3253646735 conn(0x7fcd5c0b2f00 msgr2=0x7fcd5c0b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:35.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.491+0000 7fcd61084640 1 -- 172.21.15.37:0/3253646735 shutdown_connections 2024-09-17T04:34:35.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.491+0000 7fcd61084640 1 -- 172.21.15.37:0/3253646735 wait complete. 2024-09-17T04:34:35.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.492+0000 7fcd61084640 1 Processor -- start 2024-09-17T04:34:35.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.493+0000 7fcd61084640 1 -- start start 2024-09-17T04:34:35.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.494+0000 7fcd61084640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5c06d7d0 0x7fcd5c152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:35.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.494+0000 7fcd61084640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcd5c0b15b0 0x7fcd5c153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:35.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.494+0000 7fcd61084640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd5c0b19c0 0x7fcd5c157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:35.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.494+0000 7fcd61084640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c0b4b10 con 0x7fcd5c06d7d0 2024-09-17T04:34:35.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.494+0000 7fcd61084640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c0b4990 con 0x7fcd5c0b19c0 2024-09-17T04:34:35.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.494+0000 7fcd61084640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fcd5c0b4c90 con 0x7fcd5c0b15b0 2024-09-17T04:34:35.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.494+0000 7fcd5ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5c06d7d0 0x7fcd5c152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:35.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.494+0000 7fcd5ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5c06d7d0 0x7fcd5c152fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33100/0 (socket says 172.21.15.37:33100) 2024-09-17T04:34:35.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.494+0000 7fcd5ad9d640 1 -- 172.21.15.37:0/470847902 learned_addr learned my addr 172.21.15.37:0/470847902 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:35.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.494+0000 7fcd53fff640 1 --2- 172.21.15.37:0/470847902 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcd5c0b15b0 0x7fcd5c153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:35.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.494+0000 7fcd5b59e640 1 --2- 172.21.15.37:0/470847902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd5c0b19c0 0x7fcd5c157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:35.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.494+0000 7fcd5ad9d640 1 -- 172.21.15.37:0/470847902 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcd5c0b15b0 msgr2=0x7fcd5c153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:35.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.494+0000 7fcd5ad9d640 1 --2- 172.21.15.37:0/470847902 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcd5c0b15b0 0x7fcd5c153500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:35.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.494+0000 7fcd5ad9d640 1 -- 172.21.15.37:0/470847902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd5c0b19c0 msgr2=0x7fcd5c157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:35.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.494+0000 7fcd5ad9d640 1 --2- 172.21.15.37:0/470847902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd5c0b19c0 0x7fcd5c157270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:35.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.494+0000 7fcd5ad9d640 1 -- 172.21.15.37:0/470847902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5c153a40 con 0x7fcd5c06d7d0 2024-09-17T04:34:35.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.495+0000 7fcd5ad9d640 1 --2- 172.21.15.37:0/470847902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5c06d7d0 0x7fcd5c152fc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00d770 tx=0x7fcd4c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:35.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.495+0000 7fcd537fe640 1 -- 172.21.15.37:0/470847902 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c017070 con 0x7fcd5c06d7d0 2024-09-17T04:34:35.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.495+0000 7fcd537fe640 1 -- 172.21.15.37:0/470847902 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd4c009d20 con 0x7fcd5c06d7d0 2024-09-17T04:34:35.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.495+0000 7fcd61084640 1 -- 172.21.15.37:0/470847902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd5c153db0 con 0x7fcd5c06d7d0 2024-09-17T04:34:35.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.495+0000 7fcd537fe640 1 -- 172.21.15.37:0/470847902 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c0106c0 con 0x7fcd5c06d7d0 2024-09-17T04:34:35.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.495+0000 7fcd61084640 1 -- 172.21.15.37:0/470847902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd5c154370 con 0x7fcd5c06d7d0 2024-09-17T04:34:35.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.496+0000 7fcd61084640 1 -- 172.21.15.37:0/470847902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd5c15ca00 con 0x7fcd5c06d7d0 2024-09-17T04:34:35.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.497+0000 7fcd537fe640 1 -- 172.21.15.37:0/470847902 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcd4c01a070 con 0x7fcd5c06d7d0 2024-09-17T04:34:35.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.497+0000 7fcd537fe640 1 --2- 172.21.15.37:0/470847902 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fcd3c078b90 0x7fcd3c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:35.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.497+0000 7fcd537fe640 1 -- 172.21.15.37:0/470847902 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7fcd4c09b020 con 0x7fcd5c06d7d0 2024-09-17T04:34:35.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.497+0000 7fcd53fff640 1 --2- 172.21.15.37:0/470847902 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fcd3c078b90 0x7fcd3c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:35.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.498+0000 7fcd53fff640 1 --2- 172.21.15.37:0/470847902 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fcd3c078b90 0x7fcd3c07b0a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcd44033000 tx=0x7fcd44004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:35.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.502+0000 7fcd537fe640 1 -- 172.21.15.37:0/470847902 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fcd4c062bf0 con 0x7fcd5c06d7d0 2024-09-17T04:34:35.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:35.724+0000 7fcd61084640 1 -- 172.21.15.37:0/470847902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcd5c0c4c30 con 0x7fcd5c06d7d0 2024-09-17T04:34:36.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.540+0000 7fcd537fe640 1 -- 172.21.15.37:0/470847902 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7fcd4c067620 con 0x7fcd5c06d7d0 2024-09-17T04:34:36.541 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T04:34:36.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.544+0000 7fcd61084640 1 -- 172.21.15.37:0/470847902 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fcd3c078b90 msgr2=0x7fcd3c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:36.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.544+0000 7fcd61084640 1 --2- 172.21.15.37:0/470847902 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fcd3c078b90 0x7fcd3c07b0a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcd44033000 tx=0x7fcd44004730 comp rx=0 tx=0).stop 2024-09-17T04:34:36.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.544+0000 7fcd61084640 1 -- 172.21.15.37:0/470847902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5c06d7d0 msgr2=0x7fcd5c152fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:36.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.544+0000 7fcd61084640 1 --2- 172.21.15.37:0/470847902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5c06d7d0 0x7fcd5c152fc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00d770 tx=0x7fcd4c00dc30 comp rx=0 tx=0).stop 2024-09-17T04:34:36.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.544+0000 7fcd61084640 1 -- 172.21.15.37:0/470847902 shutdown_connections 2024-09-17T04:34:36.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.544+0000 7fcd61084640 1 --2- 172.21.15.37:0/470847902 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fcd3c078b90 0x7fcd3c07b0a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:36.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.544+0000 7fcd61084640 1 --2- 172.21.15.37:0/470847902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd5c0b19c0 0x7fcd5c157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:36.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.544+0000 7fcd61084640 1 --2- 172.21.15.37:0/470847902 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcd5c0b15b0 0x7fcd5c153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:36.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.544+0000 7fcd61084640 1 --2- 172.21.15.37:0/470847902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5c06d7d0 0x7fcd5c152fc0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:36.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.544+0000 7fcd61084640 1 -- 172.21.15.37:0/470847902 >> 172.21.15.37:0/470847902 conn(0x7fcd5c0b2f00 msgr2=0x7fcd5c0b0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:36.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.544+0000 7fcd61084640 1 -- 172.21.15.37:0/470847902 shutdown_connections 2024-09-17T04:34:36.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.544+0000 7fcd61084640 1 -- 172.21.15.37:0/470847902 wait complete. 2024-09-17T04:34:36.558 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T04:34:36.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.684+0000 7f10f0d57640 1 -- 172.21.15.37:0/1128772743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10ec0b3530 msgr2=0x7f10ec06dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:36.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.684+0000 7f10f0d57640 1 --2- 172.21.15.37:0/1128772743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10ec0b3530 0x7f10ec06dea0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f10dc005fd0 tx=0x7f10dc02a5d0 comp rx=0 tx=0).stop 2024-09-17T04:34:36.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.684+0000 7f10f0d57640 1 -- 172.21.15.37:0/1128772743 shutdown_connections 2024-09-17T04:34:36.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.684+0000 7f10f0d57640 1 --2- 172.21.15.37:0/1128772743 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f10ec0b9c40 0x7f10ec0d4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:36.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.684+0000 7f10f0d57640 1 --2- 172.21.15.37:0/1128772743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10ec0b3530 0x7f10ec06dea0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:36.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.684+0000 7f10f0d57640 1 -- 172.21.15.37:0/1128772743 >> 172.21.15.37:0/1128772743 conn(0x7f10ec08bef0 msgr2=0x7f10ec0d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:36.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.684+0000 7f10f0d57640 1 -- 172.21.15.37:0/1128772743 shutdown_connections 2024-09-17T04:34:36.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.684+0000 7f10f0d57640 1 -- 172.21.15.37:0/1128772743 wait complete. 2024-09-17T04:34:36.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.686+0000 7f10f0d57640 1 Processor -- start 2024-09-17T04:34:36.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.686+0000 7f10f0d57640 1 -- start start 2024-09-17T04:34:36.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.687+0000 7f10f0d57640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10ec0b3530 0x7f10ec150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:36.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.687+0000 7f10f0d57640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f10ec0b9c40 0x7f10ec0b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:36.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.687+0000 7f10f0d57640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ec1626b0 0x7f10ec0b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:36.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.687+0000 7f10f0d57640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec0d31c0 con 0x7f10ec0b3530 2024-09-17T04:34:36.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.687+0000 7f10f0d57640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec0d3040 con 0x7f10ec1626b0 2024-09-17T04:34:36.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.687+0000 7f10f0d57640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f10ec0d3340 con 0x7f10ec0b9c40 2024-09-17T04:34:36.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.687+0000 7f10ea59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10ec0b3530 0x7f10ec150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:36.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.687+0000 7f10ea59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10ec0b3530 0x7f10ec150970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33116/0 (socket says 172.21.15.37:33116) 2024-09-17T04:34:36.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.688+0000 7f10ea59c640 1 -- 172.21.15.37:0/867328102 learned_addr learned my addr 172.21.15.37:0/867328102 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:36.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.688+0000 7f10e9d9b640 1 --2- 172.21.15.37:0/867328102 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f10ec0b9c40 0x7f10ec0b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:36.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.688+0000 7f10ead9d640 1 --2- 172.21.15.37:0/867328102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ec1626b0 0x7f10ec0b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:36.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.688+0000 7f10ea59c640 1 -- 172.21.15.37:0/867328102 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f10ec0b9c40 msgr2=0x7f10ec0b26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:36.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.688+0000 7f10ea59c640 1 --2- 172.21.15.37:0/867328102 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f10ec0b9c40 0x7f10ec0b26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:36.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.688+0000 7f10ea59c640 1 -- 172.21.15.37:0/867328102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ec1626b0 msgr2=0x7f10ec0b2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:36.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.688+0000 7f10ea59c640 1 --2- 172.21.15.37:0/867328102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ec1626b0 0x7f10ec0b2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:36.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.688+0000 7f10ea59c640 1 -- 172.21.15.37:0/867328102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10ec1f9860 con 0x7f10ec0b3530 2024-09-17T04:34:36.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.688+0000 7f10ead9d640 1 --2- 172.21.15.37:0/867328102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ec1626b0 0x7f10ec0b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:36.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.688+0000 7f10ea59c640 1 --2- 172.21.15.37:0/867328102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10ec0b3530 0x7f10ec150970 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f10dc006010 tx=0x7f10dc0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:36.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.688+0000 7f10daffd640 1 -- 172.21.15.37:0/867328102 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc03e030 con 0x7f10ec0b3530 2024-09-17T04:34:36.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.688+0000 7f10daffd640 1 -- 172.21.15.37:0/867328102 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10dc003ae0 con 0x7f10ec0b3530 2024-09-17T04:34:36.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.688+0000 7f10f0d57640 1 -- 172.21.15.37:0/867328102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10ec1f9b40 con 0x7f10ec0b3530 2024-09-17T04:34:36.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.688+0000 7f10daffd640 1 -- 172.21.15.37:0/867328102 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc035040 con 0x7f10ec0b3530 2024-09-17T04:34:36.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.688+0000 7f10f0d57640 1 -- 172.21.15.37:0/867328102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10ec1fa100 con 0x7f10ec0b3530 2024-09-17T04:34:36.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.689+0000 7f10f0d57640 1 -- 172.21.15.37:0/867328102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10ec06e200 con 0x7f10ec0b3530 2024-09-17T04:34:36.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.690+0000 7f10daffd640 1 -- 172.21.15.37:0/867328102 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f10dc041030 con 0x7f10ec0b3530 2024-09-17T04:34:36.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.691+0000 7f10daffd640 1 --2- 172.21.15.37:0/867328102 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f10cc078bc0 0x7f10cc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:36.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.691+0000 7f10daffd640 1 -- 172.21.15.37:0/867328102 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7f10dc0bb320 con 0x7f10ec0b3530 2024-09-17T04:34:36.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.691+0000 7f10e9d9b640 1 --2- 172.21.15.37:0/867328102 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f10cc078bc0 0x7f10cc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:36.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.691+0000 7f10e9d9b640 1 --2- 172.21.15.37:0/867328102 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f10cc078bc0 0x7f10cc07b0d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f10ec15ff70 tx=0x7f10e0005f30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:36.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.695+0000 7f10daffd640 1 -- 172.21.15.37:0/867328102 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f10ec06e200 con 0x7f10ec0b3530 2024-09-17T04:34:36.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:36.917+0000 7f10f0d57640 1 -- 172.21.15.37:0/867328102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f10ec0d3890 con 0x7f10ec0b3530 2024-09-17T04:34:37.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.542+0000 7f10daffd640 1 -- 172.21.15.37:0/867328102 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7f10ec0d3890 con 0x7f10ec0b3530 2024-09-17T04:34:37.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.545+0000 7f10f0d57640 1 -- 172.21.15.37:0/867328102 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f10cc078bc0 msgr2=0x7f10cc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:37.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.545+0000 7f10f0d57640 1 --2- 172.21.15.37:0/867328102 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f10cc078bc0 0x7f10cc07b0d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f10ec15ff70 tx=0x7f10e0005f30 comp rx=0 tx=0).stop 2024-09-17T04:34:37.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.545+0000 7f10f0d57640 1 -- 172.21.15.37:0/867328102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10ec0b3530 msgr2=0x7f10ec150970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:37.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.545+0000 7f10f0d57640 1 --2- 172.21.15.37:0/867328102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10ec0b3530 0x7f10ec150970 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f10dc006010 tx=0x7f10dc0035f0 comp rx=0 tx=0).stop 2024-09-17T04:34:37.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.545+0000 7f10f0d57640 1 -- 172.21.15.37:0/867328102 shutdown_connections 2024-09-17T04:34:37.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.545+0000 7f10f0d57640 1 --2- 172.21.15.37:0/867328102 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f10cc078bc0 0x7f10cc07b0d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:37.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.545+0000 7f10f0d57640 1 --2- 172.21.15.37:0/867328102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ec1626b0 0x7f10ec0b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:37.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.545+0000 7f10f0d57640 1 --2- 172.21.15.37:0/867328102 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f10ec0b9c40 0x7f10ec0b26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:37.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.545+0000 7f10f0d57640 1 --2- 172.21.15.37:0/867328102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10ec0b3530 0x7f10ec150970 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:37.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.545+0000 7f10f0d57640 1 -- 172.21.15.37:0/867328102 >> 172.21.15.37:0/867328102 conn(0x7f10ec08bef0 msgr2=0x7f10ec06f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:37.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.545+0000 7f10f0d57640 1 -- 172.21.15.37:0/867328102 shutdown_connections 2024-09-17T04:34:37.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.545+0000 7f10f0d57640 1 -- 172.21.15.37:0/867328102 wait complete. 2024-09-17T04:34:37.559 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T04:34:37.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.687+0000 7fc13e070640 1 -- 172.21.15.37:0/2165971792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1380cdd20 msgr2=0x7fc138160530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:37.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.687+0000 7fc13e070640 1 --2- 172.21.15.37:0/2165971792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1380cdd20 0x7fc138160530 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc130005fd0 tx=0x7fc13002aa90 comp rx=0 tx=0).stop 2024-09-17T04:34:37.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.687+0000 7fc13e070640 1 -- 172.21.15.37:0/2165971792 shutdown_connections 2024-09-17T04:34:37.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.688+0000 7fc13e070640 1 --2- 172.21.15.37:0/2165971792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1380cdd20 0x7fc138160530 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:37.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.688+0000 7fc13e070640 1 --2- 172.21.15.37:0/2165971792 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc1380b5220 0x7fc1380cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:37.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.688+0000 7fc13e070640 1 -- 172.21.15.37:0/2165971792 >> 172.21.15.37:0/2165971792 conn(0x7fc1380d23d0 msgr2=0x7fc1380d27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:37.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.688+0000 7fc13e070640 1 -- 172.21.15.37:0/2165971792 shutdown_connections 2024-09-17T04:34:37.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.688+0000 7fc13e070640 1 -- 172.21.15.37:0/2165971792 wait complete. 2024-09-17T04:34:37.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.689+0000 7fc13e070640 1 Processor -- start 2024-09-17T04:34:37.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.690+0000 7fc13e070640 1 -- start start 2024-09-17T04:34:37.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.691+0000 7fc13e070640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1380b5220 0x7fc1380b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:37.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.691+0000 7fc13e070640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1380cdd20 0x7fc1380b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:37.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.691+0000 7fc13e070640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc1380d2ef0 0x7fc1381f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:37.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.691+0000 7fc13e070640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc138160bf0 con 0x7fc1380b5220 2024-09-17T04:34:37.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.691+0000 7fc13e070640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc138160a70 con 0x7fc1380cdd20 2024-09-17T04:34:37.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.691+0000 7fc13e070640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fc138160d70 con 0x7fc1380d2ef0 2024-09-17T04:34:37.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.691+0000 7fc1377fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1380b5220 0x7fc1380b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:37.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.691+0000 7fc1377fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1380b5220 0x7fc1380b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33126/0 (socket says 172.21.15.37:33126) 2024-09-17T04:34:37.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.691+0000 7fc1377fe640 1 -- 172.21.15.37:0/485076961 learned_addr learned my addr 172.21.15.37:0/485076961 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:37.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.691+0000 7fc136ffd640 1 --2- 172.21.15.37:0/485076961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1380cdd20 0x7fc1380b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:37.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.691+0000 7fc137fff640 1 --2- 172.21.15.37:0/485076961 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc1380d2ef0 0x7fc1381f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:37.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.691+0000 7fc1377fe640 1 -- 172.21.15.37:0/485076961 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc1380d2ef0 msgr2=0x7fc1381f5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:37.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.691+0000 7fc1377fe640 1 --2- 172.21.15.37:0/485076961 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc1380d2ef0 0x7fc1381f5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:37.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.691+0000 7fc1377fe640 1 -- 172.21.15.37:0/485076961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1380cdd20 msgr2=0x7fc1380b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:37.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.691+0000 7fc1377fe640 1 --2- 172.21.15.37:0/485076961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1380cdd20 0x7fc1380b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:37.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.691+0000 7fc1377fe640 1 -- 172.21.15.37:0/485076961 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc13809fa70 con 0x7fc1380b5220 2024-09-17T04:34:37.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.692+0000 7fc1377fe640 1 --2- 172.21.15.37:0/485076961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1380b5220 0x7fc1380b1590 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc12800c3d0 tx=0x7fc12800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:37.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.692+0000 7fc11ffff640 1 -- 172.21.15.37:0/485076961 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc128017070 con 0x7fc1380b5220 2024-09-17T04:34:37.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.692+0000 7fc11ffff640 1 -- 172.21.15.37:0/485076961 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1280053c0 con 0x7fc1380b5220 2024-09-17T04:34:37.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.692+0000 7fc13e070640 1 -- 172.21.15.37:0/485076961 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc13809fd50 con 0x7fc1380b5220 2024-09-17T04:34:37.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.692+0000 7fc11ffff640 1 -- 172.21.15.37:0/485076961 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc128003850 con 0x7fc1380b5220 2024-09-17T04:34:37.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.692+0000 7fc13e070640 1 -- 172.21.15.37:0/485076961 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc13806f5a0 con 0x7fc1380b5220 2024-09-17T04:34:37.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.693+0000 7fc13e070640 1 -- 172.21.15.37:0/485076961 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1381612b0 con 0x7fc1380b5220 2024-09-17T04:34:37.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.694+0000 7fc11ffff640 1 -- 172.21.15.37:0/485076961 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc12801a070 con 0x7fc1380b5220 2024-09-17T04:34:37.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.694+0000 7fc11ffff640 1 --2- 172.21.15.37:0/485076961 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc124078b90 0x7fc12407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:37.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.694+0000 7fc11ffff640 1 -- 172.21.15.37:0/485076961 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6248+0+0 (secure 0 0 0) 0x7fc12809a1d0 con 0x7fc1380b5220 2024-09-17T04:34:37.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.694+0000 7fc136ffd640 1 --2- 172.21.15.37:0/485076961 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc124078b90 0x7fc12407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:37.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.695+0000 7fc136ffd640 1 --2- 172.21.15.37:0/485076961 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc124078b90 0x7fc12407b0a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc1380b6b10 tx=0x7fc12c00b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:37.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.699+0000 7fc11ffff640 1 -- 172.21.15.37:0/485076961 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc128061e30 con 0x7fc1380b5220 2024-09-17T04:34:37.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:37.883+0000 7fc13e070640 1 -- 172.21.15.37:0/485076961 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc1380b3c70 con 0x7fc1380b5220 2024-09-17T04:34:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.551+0000 7fc11ffff640 1 -- 172.21.15.37:0/485076961 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v42) ==== 181+0+0 (secure 0 0 0) 0x7fc128066860 con 0x7fc1380b5220 2024-09-17T04:34:38.552 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-09-17T04:34:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.555+0000 7fc13e070640 1 -- 172.21.15.37:0/485076961 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc124078b90 msgr2=0x7fc12407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.555+0000 7fc13e070640 1 --2- 172.21.15.37:0/485076961 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc124078b90 0x7fc12407b0a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc1380b6b10 tx=0x7fc12c00b2e0 comp rx=0 tx=0).stop 2024-09-17T04:34:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.555+0000 7fc13e070640 1 -- 172.21.15.37:0/485076961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1380b5220 msgr2=0x7fc1380b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.555+0000 7fc13e070640 1 --2- 172.21.15.37:0/485076961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1380b5220 0x7fc1380b1590 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc12800c3d0 tx=0x7fc12800c890 comp rx=0 tx=0).stop 2024-09-17T04:34:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.555+0000 7fc13e070640 1 -- 172.21.15.37:0/485076961 shutdown_connections 2024-09-17T04:34:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.555+0000 7fc13e070640 1 --2- 172.21.15.37:0/485076961 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc124078b90 0x7fc12407b0a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.555+0000 7fc13e070640 1 --2- 172.21.15.37:0/485076961 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc1380d2ef0 0x7fc1381f5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.555+0000 7fc13e070640 1 --2- 172.21.15.37:0/485076961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1380cdd20 0x7fc1380b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.555+0000 7fc13e070640 1 --2- 172.21.15.37:0/485076961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1380b5220 0x7fc1380b1590 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.555+0000 7fc13e070640 1 -- 172.21.15.37:0/485076961 >> 172.21.15.37:0/485076961 conn(0x7fc1380d23d0 msgr2=0x7fc1380d27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.555+0000 7fc13e070640 1 -- 172.21.15.37:0/485076961 shutdown_connections 2024-09-17T04:34:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.555+0000 7fc13e070640 1 -- 172.21.15.37:0/485076961 wait complete. 2024-09-17T04:34:38.569 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T04:34:38.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.697+0000 7f66a6615640 1 -- 172.21.15.37:0/2248798030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f668c004800 msgr2=0x7f668c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:38.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.697+0000 7f66a6615640 1 --2- 172.21.15.37:0/2248798030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f668c004800 0x7f668c004c10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6688009b70 tx=0x7f668c005830 comp rx=0 tx=0).stop 2024-09-17T04:34:38.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.697+0000 7f66a4bb2640 1 -- 172.21.15.37:0/2248798030 reap_dead start 2024-09-17T04:34:38.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.697+0000 7f66a6615640 1 -- 172.21.15.37:0/2248798030 shutdown_connections 2024-09-17T04:34:38.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.697+0000 7f66a6615640 1 -- 172.21.15.37:0/2248798030 >> 172.21.15.37:0/2248798030 conn(0x7f66a008bef0 msgr2=0x7f66a00d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:38.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.697+0000 7f66a6615640 1 -- 172.21.15.37:0/2248798030 shutdown_connections 2024-09-17T04:34:38.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.697+0000 7f66a6615640 1 -- 172.21.15.37:0/2248798030 wait complete. 2024-09-17T04:34:38.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.699+0000 7f66a6615640 1 Processor -- start 2024-09-17T04:34:38.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.699+0000 7f66a6615640 1 -- start start 2024-09-17T04:34:38.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.700+0000 7f66a6615640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f668c004800 0x7f66a01f58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:38.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.700+0000 7f66a6615640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a00ce720 0x7f66a00ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:38.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.700+0000 7f66a6615640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66a00a1c60 0x7f66a00c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:38.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.700+0000 7f66a6615640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f66a0162380 con 0x7f66a00a1c60 2024-09-17T04:34:38.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.700+0000 7f66a6615640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f66a0162200 con 0x7f66a00ce720 2024-09-17T04:34:38.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.700+0000 7f66a6615640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f66a00d2850 con 0x7f668c004800 2024-09-17T04:34:38.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.700+0000 7f66a4bb2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66a00a1c60 0x7f66a00c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:38.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.701+0000 7f66a4bb2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66a00a1c60 0x7f66a00c8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33140/0 (socket says 172.21.15.37:33140) 2024-09-17T04:34:38.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.701+0000 7f66a4bb2640 1 -- 172.21.15.37:0/2883483705 learned_addr learned my addr 172.21.15.37:0/2883483705 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:38.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.701+0000 7f669ffff640 1 --2- 172.21.15.37:0/2883483705 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f668c004800 0x7f66a01f58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:38.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.701+0000 7f6697fff640 1 --2- 172.21.15.37:0/2883483705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a00ce720 0x7f66a00ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:38.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.701+0000 7f66a4bb2640 1 -- 172.21.15.37:0/2883483705 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f668c004800 msgr2=0x7f66a01f58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:38.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.701+0000 7f66a4bb2640 1 --2- 172.21.15.37:0/2883483705 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f668c004800 0x7f66a01f58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:38.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.701+0000 7f66a4bb2640 1 -- 172.21.15.37:0/2883483705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a00ce720 msgr2=0x7f66a00ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:38.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.701+0000 7f66a4bb2640 1 --2- 172.21.15.37:0/2883483705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a00ce720 0x7f66a00ca850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:38.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.701+0000 7f66a4bb2640 1 -- 172.21.15.37:0/2883483705 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a00cc260 con 0x7f66a00a1c60 2024-09-17T04:34:38.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.701+0000 7f6697fff640 1 --2- 172.21.15.37:0/2883483705 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66a00ce720 0x7f66a00ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:38.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.701+0000 7f66a4bb2640 1 --2- 172.21.15.37:0/2883483705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66a00a1c60 0x7f66a00c8020 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f66a00b7130 tx=0x7f669800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:38.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.701+0000 7f669d7fa640 1 -- 172.21.15.37:0/2883483705 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698003d80 con 0x7f66a00a1c60 2024-09-17T04:34:38.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.701+0000 7f669d7fa640 1 -- 172.21.15.37:0/2883483705 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6698004b20 con 0x7f66a00a1c60 2024-09-17T04:34:38.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.701+0000 7f66a6615640 1 -- 172.21.15.37:0/2883483705 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a00cc540 con 0x7f66a00a1c60 2024-09-17T04:34:38.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.701+0000 7f669d7fa640 1 -- 172.21.15.37:0/2883483705 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698004030 con 0x7f66a00a1c60 2024-09-17T04:34:38.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.701+0000 7f66a6615640 1 -- 172.21.15.37:0/2883483705 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a01f53d0 con 0x7f66a00a1c60 2024-09-17T04:34:38.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.702+0000 7f66a6615640 1 -- 172.21.15.37:0/2883483705 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a00b34c0 con 0x7f66a00a1c60 2024-09-17T04:34:38.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.703+0000 7f669d7fa640 1 -- 172.21.15.37:0/2883483705 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f66980041d0 con 0x7f66a00a1c60 2024-09-17T04:34:38.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.704+0000 7f669d7fa640 1 --2- 172.21.15.37:0/2883483705 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6680078b90 0x7f668007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:38.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.704+0000 7f669d7fa640 1 -- 172.21.15.37:0/2883483705 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5810+0+0 (secure 0 0 0) 0x7f669809bbe0 con 0x7f66a00a1c60 2024-09-17T04:34:38.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.704+0000 7f669ffff640 1 --2- 172.21.15.37:0/2883483705 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6680078b90 0x7f668007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:38.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.705+0000 7f669ffff640 1 --2- 172.21.15.37:0/2883483705 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f6680078b90 0x7f668007b0a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6690001010 tx=0x7f6690000f80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:38.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.709+0000 7f669d7fa640 1 -- 172.21.15.37:0/2883483705 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6698064040 con 0x7f66a00a1c60 2024-09-17T04:34:38.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.897+0000 7f66a6615640 1 -- 172.21.15.37:0/2883483705 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f66a00b97f0 con 0x7f66a00a1c60 2024-09-17T04:34:38.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:38.898+0000 7f669d7fa640 1 -- 172.21.15.37:0/2883483705 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=-16 pool 'cephfs_data' is in use by CephFS v42) ==== 181+0+0 (secure 0 0 0) 0x7f6698063940 con 0x7f66a00a1c60 2024-09-17T04:34:38.898 INFO:teuthology.orchestra.run.smithi037.stderr:Error EBUSY: pool 'cephfs_data' is in use by CephFS 2024-09-17T04:34:38.903 DEBUG:teuthology.orchestra.run:got remote process result: 16 2024-09-17T04:34:38.904 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:34:39.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.021+0000 7f03d1e15640 1 -- 172.21.15.37:0/2386368120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03b40040b0 msgr2=0x7f03b4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:39.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.021+0000 7f03d1e15640 1 --2- 172.21.15.37:0/2386368120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03b40040b0 0x7f03b4004800 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009b10 tx=0x7f03b4005830 comp rx=0 tx=0).stop 2024-09-17T04:34:39.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.021+0000 7f03cbfff640 1 -- 172.21.15.37:0/2386368120 reap_dead start 2024-09-17T04:34:39.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.021+0000 7f03d1e15640 1 -- 172.21.15.37:0/2386368120 shutdown_connections 2024-09-17T04:34:39.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.021+0000 7f03d1e15640 1 -- 172.21.15.37:0/2386368120 >> 172.21.15.37:0/2386368120 conn(0x7f03cc08bef0 msgr2=0x7f03cc0d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:39.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.021+0000 7f03d1e15640 1 -- 172.21.15.37:0/2386368120 shutdown_connections 2024-09-17T04:34:39.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.021+0000 7f03d1e15640 1 -- 172.21.15.37:0/2386368120 wait complete. 2024-09-17T04:34:39.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.023+0000 7f03d1e15640 1 Processor -- start 2024-09-17T04:34:39.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.023+0000 7f03d1e15640 1 -- start start 2024-09-17T04:34:39.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.024+0000 7f03d1e15640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f03b40040b0 0x7f03cc0b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:39.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.024+0000 7f03d1e15640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03cc14ad30 0x7f03cc1f87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:39.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.024+0000 7f03d1e15640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03cc1f8d30 0x7f03cc1f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:39.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.024+0000 7f03d1e15640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc0b3b90 con 0x7f03cc14ad30 2024-09-17T04:34:39.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.024+0000 7f03d1e15640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc0d28f0 con 0x7f03cc1f8d30 2024-09-17T04:34:39.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.024+0000 7f03d1e15640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f03cc0b94e0 con 0x7f03b40040b0 2024-09-17T04:34:39.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.024+0000 7f03caffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03cc14ad30 0x7f03cc1f87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:39.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.024+0000 7f03caffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03cc14ad30 0x7f03cc1f87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33144/0 (socket says 172.21.15.37:33144) 2024-09-17T04:34:39.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.024+0000 7f03caffd640 1 -- 172.21.15.37:0/356770978 learned_addr learned my addr 172.21.15.37:0/356770978 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:39.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.024+0000 7f03cb7fe640 1 --2- 172.21.15.37:0/356770978 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f03b40040b0 0x7f03cc0b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:39.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.024+0000 7f03cbfff640 1 --2- 172.21.15.37:0/356770978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03cc1f8d30 0x7f03cc1f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:39.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.024+0000 7f03caffd640 1 -- 172.21.15.37:0/356770978 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f03b40040b0 msgr2=0x7f03cc0b7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:39.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.024+0000 7f03caffd640 1 --2- 172.21.15.37:0/356770978 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f03b40040b0 0x7f03cc0b7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:39.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.025+0000 7f03caffd640 1 -- 172.21.15.37:0/356770978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03cc1f8d30 msgr2=0x7f03cc1f63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:39.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.025+0000 7f03caffd640 1 --2- 172.21.15.37:0/356770978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03cc1f8d30 0x7f03cc1f63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:39.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.025+0000 7f03caffd640 1 -- 172.21.15.37:0/356770978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03cc1f9140 con 0x7f03cc14ad30 2024-09-17T04:34:39.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.025+0000 7f03caffd640 1 --2- 172.21.15.37:0/356770978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03cc14ad30 0x7f03cc1f87f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009a60 tx=0x7f03c00336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:39.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.025+0000 7f03b3fff640 1 -- 172.21.15.37:0/356770978 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0038040 con 0x7f03cc14ad30 2024-09-17T04:34:39.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.025+0000 7f03d1e15640 1 -- 172.21.15.37:0/356770978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03cc0b8440 con 0x7f03cc14ad30 2024-09-17T04:34:39.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.025+0000 7f03b3fff640 1 -- 172.21.15.37:0/356770978 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03c0041070 con 0x7f03cc14ad30 2024-09-17T04:34:39.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.025+0000 7f03b3fff640 1 -- 172.21.15.37:0/356770978 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0033c50 con 0x7f03cc14ad30 2024-09-17T04:34:39.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.025+0000 7f03d1e15640 1 -- 172.21.15.37:0/356770978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03cc0b8a00 con 0x7f03cc14ad30 2024-09-17T04:34:39.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.026+0000 7f03d1e15640 1 -- 172.21.15.37:0/356770978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03cc06a150 con 0x7f03cc14ad30 2024-09-17T04:34:39.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.027+0000 7f03b3fff640 1 -- 172.21.15.37:0/356770978 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f03c0046460 con 0x7f03cc14ad30 2024-09-17T04:34:39.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.027+0000 7f03b3fff640 1 --2- 172.21.15.37:0/356770978 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f03b8078b90 0x7f03b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:39.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.028+0000 7f03b3fff640 1 -- 172.21.15.37:0/356770978 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5810+0+0 (secure 0 0 0) 0x7f03c00bba70 con 0x7f03cc14ad30 2024-09-17T04:34:39.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.028+0000 7f03cb7fe640 1 --2- 172.21.15.37:0/356770978 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f03b8078b90 0x7f03b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:39.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.028+0000 7f03cb7fe640 1 --2- 172.21.15.37:0/356770978 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f03b8078b90 0x7f03b807b0a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f03bc000ff0 tx=0x7f03bc001bb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:39.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.032+0000 7f03b3fff640 1 -- 172.21.15.37:0/356770978 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f03cc1f9140 con 0x7f03cc14ad30 2024-09-17T04:34:39.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.262+0000 7f03d1e15640 1 -- 172.21.15.37:0/356770978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03cc0b81d0 con 0x7f03cc14ad30 2024-09-17T04:34:39.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.263+0000 7f03b3fff640 1 -- 172.21.15.37:0/356770978 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5223 (secure 0 0 0) 0x7f03c0083d20 con 0x7f03cc14ad30 2024-09-17T04:34:39.263 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:39.263 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"btime":"2024-09-17T04:34:37:541562+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/199192833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":199192833},{"type":"v1","addr":"172.21.15.37:6833","nonce":199192833}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4832,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/4098913371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4098913371},{"type":"v1","addr":"172.21.15.92:6836","nonce":4098913371}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/383658044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":383658044},{"type":"v1","addr":"172.21.15.92:6837","nonce":383658044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:34:17.928780+0000","modified":"2024-09-17T04:34:28.244429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5208":{"gid":5208,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6835/4051839052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":4051839052},{"type":"v1","addr":"172.21.15.37:6835","nonce":4051839052}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":6,"enabled":true,"fs_name":"recovery_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5208,"qdb_cluster":[5208]},"id":3}]} 2024-09-17T04:34:39.263 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 29 2024-09-17T04:34:39.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.265+0000 7f03d1e15640 1 -- 172.21.15.37:0/356770978 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f03b8078b90 msgr2=0x7f03b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:39.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.265+0000 7f03d1e15640 1 --2- 172.21.15.37:0/356770978 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f03b8078b90 0x7f03b807b0a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f03bc000ff0 tx=0x7f03bc001bb0 comp rx=0 tx=0).stop 2024-09-17T04:34:39.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.265+0000 7f03d1e15640 1 -- 172.21.15.37:0/356770978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03cc14ad30 msgr2=0x7f03cc1f87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:39.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.265+0000 7f03d1e15640 1 --2- 172.21.15.37:0/356770978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03cc14ad30 0x7f03cc1f87f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009a60 tx=0x7f03c00336f0 comp rx=0 tx=0).stop 2024-09-17T04:34:39.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.265+0000 7f03d1e15640 1 -- 172.21.15.37:0/356770978 shutdown_connections 2024-09-17T04:34:39.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.265+0000 7f03d1e15640 1 --2- 172.21.15.37:0/356770978 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f03b8078b90 0x7f03b807b0a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:39.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.265+0000 7f03d1e15640 1 --2- 172.21.15.37:0/356770978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03cc1f8d30 0x7f03cc1f63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:39.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.265+0000 7f03d1e15640 1 --2- 172.21.15.37:0/356770978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03cc14ad30 0x7f03cc1f87f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:39.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.265+0000 7f03d1e15640 1 --2- 172.21.15.37:0/356770978 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f03b40040b0 0x7f03cc0b7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:39.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.265+0000 7f03d1e15640 1 -- 172.21.15.37:0/356770978 >> 172.21.15.37:0/356770978 conn(0x7f03cc08bef0 msgr2=0x7f03cc161750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:39.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.265+0000 7f03d1e15640 1 -- 172.21.15.37:0/356770978 shutdown_connections 2024-09-17T04:34:39.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.265+0000 7f03d1e15640 1 -- 172.21.15.37:0/356770978 wait complete. 2024-09-17T04:34:39.276 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:34:39.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.392+0000 7fece44de640 1 -- 172.21.15.37:0/2534240491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecdc0ce600 msgr2=0x7fecdc0b1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:39.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.392+0000 7fece44de640 1 --2- 172.21.15.37:0/2534240491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecdc0ce600 0x7fecdc0b1560 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0d0df0 tx=0x7fecd802a530 comp rx=0 tx=0).stop 2024-09-17T04:34:39.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.392+0000 7fece44de640 1 -- 172.21.15.37:0/2534240491 shutdown_connections 2024-09-17T04:34:39.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.392+0000 7fece44de640 1 --2- 172.21.15.37:0/2534240491 >> v2:172.21.15.92:3300/0 conn(0x7fecdc0d1e50 0x7fecdc0cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:39.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.392+0000 7fece44de640 1 --2- 172.21.15.37:0/2534240491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecdc0ce600 0x7fecdc0b1560 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:39.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.392+0000 7fece44de640 1 -- 172.21.15.37:0/2534240491 >> 172.21.15.37:0/2534240491 conn(0x7fecdc0d1330 msgr2=0x7fecdc0d1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:39.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.392+0000 7fece44de640 1 -- 172.21.15.37:0/2534240491 shutdown_connections 2024-09-17T04:34:39.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.392+0000 7fece44de640 1 -- 172.21.15.37:0/2534240491 wait complete. 2024-09-17T04:34:39.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.394+0000 7fece44de640 1 Processor -- start 2024-09-17T04:34:39.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.394+0000 7fece44de640 1 -- start start 2024-09-17T04:34:39.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.395+0000 7fece44de640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecdc0ce600 0x7fecdc14b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:39.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.395+0000 7fece44de640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fecdc0cfd30 0x7fecdc06cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:39.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.395+0000 7fece44de640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecdc0d1e50 0x7fecdc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:39.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.395+0000 7fece44de640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc15ea60 con 0x7fecdc0d1e50 2024-09-17T04:34:39.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.395+0000 7fece44de640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc15e8e0 con 0x7fecdc0ce600 2024-09-17T04:34:39.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.395+0000 7fece44de640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fecdc15ebe0 con 0x7fecdc0cfd30 2024-09-17T04:34:39.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.395+0000 7fece2a7b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecdc0d1e50 0x7fecdc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:39.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.395+0000 7fece2a7b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecdc0d1e50 0x7fecdc06d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33176/0 (socket says 172.21.15.37:33176) 2024-09-17T04:34:39.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.395+0000 7fece2a7b640 1 -- 172.21.15.37:0/928823086 learned_addr learned my addr 172.21.15.37:0/928823086 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:39.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.396+0000 7fece227a640 1 --2- 172.21.15.37:0/928823086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecdc0ce600 0x7fecdc14b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:39.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.396+0000 7fece1a79640 1 --2- 172.21.15.37:0/928823086 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fecdc0cfd30 0x7fecdc06cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:39.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.396+0000 7fece2a7b640 1 -- 172.21.15.37:0/928823086 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fecdc0cfd30 msgr2=0x7fecdc06cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:39.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.396+0000 7fece2a7b640 1 --2- 172.21.15.37:0/928823086 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fecdc0cfd30 0x7fecdc06cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:39.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.396+0000 7fece2a7b640 1 -- 172.21.15.37:0/928823086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecdc0ce600 msgr2=0x7fecdc14b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:39.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.396+0000 7fece2a7b640 1 --2- 172.21.15.37:0/928823086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecdc0ce600 0x7fecdc14b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:39.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.396+0000 7fece2a7b640 1 -- 172.21.15.37:0/928823086 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecdc14bc90 con 0x7fecdc0d1e50 2024-09-17T04:34:39.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.396+0000 7fece2a7b640 1 --2- 172.21.15.37:0/928823086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecdc0d1e50 0x7fecdc06d300 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fecd400c730 tx=0x7fecd400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:39.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.396+0000 7fecd2ffd640 1 -- 172.21.15.37:0/928823086 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd4003b70 con 0x7fecdc0d1e50 2024-09-17T04:34:39.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.396+0000 7fecd2ffd640 1 -- 172.21.15.37:0/928823086 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecd4003d30 con 0x7fecdc0d1e50 2024-09-17T04:34:39.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.396+0000 7fece44de640 1 -- 172.21.15.37:0/928823086 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecdc15b550 con 0x7fecdc0d1e50 2024-09-17T04:34:39.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.396+0000 7fecd2ffd640 1 -- 172.21.15.37:0/928823086 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd40052d0 con 0x7fecdc0d1e50 2024-09-17T04:34:39.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.396+0000 7fece44de640 1 -- 172.21.15.37:0/928823086 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecdc1f7140 con 0x7fecdc0d1e50 2024-09-17T04:34:39.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.397+0000 7fece44de640 1 -- 172.21.15.37:0/928823086 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecdc15f100 con 0x7fecdc0d1e50 2024-09-17T04:34:39.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.398+0000 7fecd2ffd640 1 -- 172.21.15.37:0/928823086 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fecd4005470 con 0x7fecdc0d1e50 2024-09-17T04:34:39.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.398+0000 7fecd2ffd640 1 --2- 172.21.15.37:0/928823086 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fecc0078b90 0x7fecc007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:39.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.399+0000 7fecd2ffd640 1 -- 172.21.15.37:0/928823086 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5810+0+0 (secure 0 0 0) 0x7fecd409ac30 con 0x7fecdc0d1e50 2024-09-17T04:34:39.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.399+0000 7fece227a640 1 --2- 172.21.15.37:0/928823086 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fecc0078b90 0x7fecc007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:39.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.399+0000 7fece227a640 1 --2- 172.21.15.37:0/928823086 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fecc0078b90 0x7fecc007b0a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fecd8034000 tx=0x7fecd8004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:39.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.403+0000 7fecd2ffd640 1 -- 172.21.15.37:0/928823086 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fecd4062f80 con 0x7fecdc0d1e50 2024-09-17T04:34:39.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.590+0000 7fece44de640 1 -- 172.21.15.37:0/928823086 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fecdc09f350 con 0x7fecdc0d1e50 2024-09-17T04:34:39.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.591+0000 7fecd2ffd640 1 -- 172.21.15.37:0/928823086 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16813 (secure 0 0 0) 0x7fecd4062880 con 0x7fecdc0d1e50 2024-09-17T04:34:39.592 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:39.592 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":42,"fsid":"a908a603-430c-4209-8294-0c3460b35f42","created":"2024-09-17T04:32:54.139221+0000","modified":"2024-09-17T04:34:38.547869+0000","last_up_change":"2024-09-17T04:33:06.922210+0000","last_in_change":"2024-09-17T04:32:57.754118+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:33:08.461907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:58.446650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"recovery_fs_metadata","create_time":"2024-09-17T04:34:16.436885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ea4b8d1-0d22-4dd9-b913-9df585ef24ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6809","nonce":2283587593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6811","nonce":2283587593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6815","nonce":2283587593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6813","nonce":2283587593}]},"public_addr":"172.21.15.37:6809/2283587593","cluster_addr":"172.21.15.37:6811/2283587593","heartbeat_back_addr":"172.21.15.37:6815/2283587593","heartbeat_front_addr":"172.21.15.37:6813/2283587593","state":["exists","up"]},{"osd":1,"uuid":"ac745a33-e8ad-4083-9c18-71bf81982b76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6825","nonce":2657955935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6827","nonce":2657955935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6831","nonce":2657955935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6829","nonce":2657955935}]},"public_addr":"172.21.15.37:6825/2657955935","cluster_addr":"172.21.15.37:6827/2657955935","heartbeat_back_addr":"172.21.15.37:6831/2657955935","heartbeat_front_addr":"172.21.15.37:6829/2657955935","state":["exists","up"]},{"osd":2,"uuid":"3c63f3d8-132a-4735-b0bc-d64313a46503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6817","nonce":579720176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6819","nonce":579720176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6823","nonce":579720176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6821","nonce":579720176}]},"public_addr":"172.21.15.37:6817/579720176","cluster_addr":"172.21.15.37:6819/579720176","heartbeat_back_addr":"172.21.15.37:6823/579720176","heartbeat_front_addr":"172.21.15.37:6821/579720176","state":["exists","up"]},{"osd":3,"uuid":"4f8c6ca2-ea1d-4610-9127-52c4dd8bc8e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6801","nonce":809975826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6803","nonce":809975826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6807","nonce":809975826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6805","nonce":809975826}]},"public_addr":"172.21.15.37:6801/809975826","cluster_addr":"172.21.15.37:6803/809975826","heartbeat_back_addr":"172.21.15.37:6807/809975826","heartbeat_front_addr":"172.21.15.37:6805/809975826","state":["exists","up"]},{"osd":4,"uuid":"63d0c4ab-f115-47b4-9611-12a10153473a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6819","nonce":2509785752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6821","nonce":2509785752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6825","nonce":2509785752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6823","nonce":2509785752}]},"public_addr":"172.21.15.92:6819/2509785752","cluster_addr":"172.21.15.92:6821/2509785752","heartbeat_back_addr":"172.21.15.92:6825/2509785752","heartbeat_front_addr":"172.21.15.92:6823/2509785752","state":["exists","up"]},{"osd":5,"uuid":"4c56c04d-0c07-4f76-9d75-6335121310ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6827","nonce":248263832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6829","nonce":248263832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6833","nonce":248263832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6831","nonce":248263832}]},"public_addr":"172.21.15.92:6827/248263832","cluster_addr":"172.21.15.92:6829/248263832","heartbeat_back_addr":"172.21.15.92:6833/248263832","heartbeat_front_addr":"172.21.15.92:6831/248263832","state":["exists","up"]},{"osd":6,"uuid":"b30eae7c-096a-4d24-ac51-96b8b13ecaee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6811","nonce":4078365985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6813","nonce":4078365985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6817","nonce":4078365985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6815","nonce":4078365985}]},"public_addr":"172.21.15.92:6811/4078365985","cluster_addr":"172.21.15.92:6813/4078365985","heartbeat_back_addr":"172.21.15.92:6817/4078365985","heartbeat_front_addr":"172.21.15.92:6815/4078365985","state":["exists","up"]},{"osd":7,"uuid":"62f12540-d374-4a4c-9ff7-63b7304fe77f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6803","nonce":3444787223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6805","nonce":3444787223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6809","nonce":3444787223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6807","nonce":3444787223}]},"public_addr":"172.21.15.92:6803/3444787223","cluster_addr":"172.21.15.92:6805/3444787223","heartbeat_back_addr":"172.21.15.92:6809/3444787223","heartbeat_front_addr":"172.21.15.92:6807/3444787223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.006420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.111135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.103325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.072656+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.200989+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.060455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.774773+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.865778+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/1358600605":"2024-09-18T04:34:15.699884+0000","172.21.15.37:6833/1358600605":"2024-09-18T04:34:15.699884+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:34:39.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.594+0000 7fece44de640 1 -- 172.21.15.37:0/928823086 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fecc0078b90 msgr2=0x7fecc007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:39.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.594+0000 7fece44de640 1 --2- 172.21.15.37:0/928823086 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fecc0078b90 0x7fecc007b0a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fecd8034000 tx=0x7fecd8004730 comp rx=0 tx=0).stop 2024-09-17T04:34:39.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.594+0000 7fece44de640 1 -- 172.21.15.37:0/928823086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecdc0d1e50 msgr2=0x7fecdc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:39.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.594+0000 7fece44de640 1 --2- 172.21.15.37:0/928823086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecdc0d1e50 0x7fecdc06d300 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fecd400c730 tx=0x7fecd400cbf0 comp rx=0 tx=0).stop 2024-09-17T04:34:39.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.594+0000 7fece44de640 1 -- 172.21.15.37:0/928823086 shutdown_connections 2024-09-17T04:34:39.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.594+0000 7fece44de640 1 --2- 172.21.15.37:0/928823086 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fecc0078b90 0x7fecc007b0a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:39.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.594+0000 7fece44de640 1 --2- 172.21.15.37:0/928823086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecdc0d1e50 0x7fecdc06d300 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:39.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.594+0000 7fece44de640 1 --2- 172.21.15.37:0/928823086 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fecdc0cfd30 0x7fecdc06cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:39.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.594+0000 7fece44de640 1 --2- 172.21.15.37:0/928823086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecdc0ce600 0x7fecdc14b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:39.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.594+0000 7fece44de640 1 -- 172.21.15.37:0/928823086 >> 172.21.15.37:0/928823086 conn(0x7fecdc0d1330 msgr2=0x7fecdc0d1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:39.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.594+0000 7fece44de640 1 -- 172.21.15.37:0/928823086 shutdown_connections 2024-09-17T04:34:39.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.594+0000 7fece44de640 1 -- 172.21.15.37:0/928823086 wait complete. 2024-09-17T04:34:39.605 INFO:tasks.cephfs.filesystem:Destroying file system recovery_fs and related pools 2024-09-17T04:34:39.605 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:34:39.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.722+0000 7f392904d640 1 -- 172.21.15.37:0/1499524857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f392406f9f0 msgr2=0x7f39240d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:39.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.722+0000 7f392904d640 1 --2- 172.21.15.37:0/1499524857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f392406f9f0 0x7f39240d0f00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3914005fd0 tx=0x7f391402a5d0 comp rx=0 tx=0).stop 2024-09-17T04:34:39.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.723+0000 7f392904d640 1 -- 172.21.15.37:0/1499524857 shutdown_connections 2024-09-17T04:34:39.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.723+0000 7f392904d640 1 --2- 172.21.15.37:0/1499524857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f392406f9f0 0x7f39240d0f00 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:39.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.723+0000 7f392904d640 1 -- 172.21.15.37:0/1499524857 >> 172.21.15.37:0/1499524857 conn(0x7f392408bef0 msgr2=0x7f39240d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:39.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.723+0000 7f392904d640 1 -- 172.21.15.37:0/1499524857 shutdown_connections 2024-09-17T04:34:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.723+0000 7f392904d640 1 -- 172.21.15.37:0/1499524857 wait complete. 2024-09-17T04:34:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.724+0000 7f392904d640 1 Processor -- start 2024-09-17T04:34:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.725+0000 7f392904d640 1 -- start start 2024-09-17T04:34:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.726+0000 7f392904d640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f392406f9f0 0x7f3924157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.726+0000 7f392904d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39240b3430 0x7f39241f67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.726+0000 7f392904d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39240d1440 0x7f392415f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.726+0000 7f392904d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f39240b3980 con 0x7f39240b3430 2024-09-17T04:34:39.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.726+0000 7f392904d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f39240d4d40 con 0x7f39240d1440 2024-09-17T04:34:39.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.726+0000 7f392904d640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f39240b3b00 con 0x7f392406f9f0 2024-09-17T04:34:39.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.726+0000 7f391bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39240b3430 0x7f39241f67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:39.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.726+0000 7f391bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39240b3430 0x7f39241f67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33182/0 (socket says 172.21.15.37:33182) 2024-09-17T04:34:39.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.726+0000 7f391bfff640 1 -- 172.21.15.37:0/4107058678 learned_addr learned my addr 172.21.15.37:0/4107058678 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:39.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.726+0000 7f3922d9d640 1 --2- 172.21.15.37:0/4107058678 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f392406f9f0 0x7f3924157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:39.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.726+0000 7f392359e640 1 --2- 172.21.15.37:0/4107058678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39240d1440 0x7f392415f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:39.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.726+0000 7f391bfff640 1 -- 172.21.15.37:0/4107058678 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f392406f9f0 msgr2=0x7f3924157ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:39.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.726+0000 7f391bfff640 1 --2- 172.21.15.37:0/4107058678 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f392406f9f0 0x7f3924157ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:39.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.726+0000 7f391bfff640 1 -- 172.21.15.37:0/4107058678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39240d1440 msgr2=0x7f392415f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:39.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.726+0000 7f391bfff640 1 --2- 172.21.15.37:0/4107058678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39240d1440 0x7f392415f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:39.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.726+0000 7f391bfff640 1 -- 172.21.15.37:0/4107058678 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39241f6d20 con 0x7f39240b3430 2024-09-17T04:34:39.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.726+0000 7f3922d9d640 1 --2- 172.21.15.37:0/4107058678 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f392406f9f0 0x7f3924157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:39.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.726+0000 7f392359e640 1 --2- 172.21.15.37:0/4107058678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39240d1440 0x7f392415f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:39.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.726+0000 7f391bfff640 1 --2- 172.21.15.37:0/4107058678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39240b3430 0x7f39241f67e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f390c001eb0 tx=0x7f390c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:39.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.727+0000 7f391b7fe640 1 -- 172.21.15.37:0/4107058678 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c017070 con 0x7f39240b3430 2024-09-17T04:34:39.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.727+0000 7f391b7fe640 1 -- 172.21.15.37:0/4107058678 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f390c0035f0 con 0x7f39240b3430 2024-09-17T04:34:39.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.727+0000 7f392904d640 1 -- 172.21.15.37:0/4107058678 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f392415b990 con 0x7f39240b3430 2024-09-17T04:34:39.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.727+0000 7f391b7fe640 1 -- 172.21.15.37:0/4107058678 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c003a00 con 0x7f39240b3430 2024-09-17T04:34:39.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.727+0000 7f392904d640 1 -- 172.21.15.37:0/4107058678 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3924159390 con 0x7f39240b3430 2024-09-17T04:34:39.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.728+0000 7f392904d640 1 -- 172.21.15.37:0/4107058678 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39240d3550 con 0x7f39240b3430 2024-09-17T04:34:39.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.728+0000 7f391b7fe640 1 -- 172.21.15.37:0/4107058678 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f390c01a070 con 0x7f39240b3430 2024-09-17T04:34:39.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.729+0000 7f391b7fe640 1 --2- 172.21.15.37:0/4107058678 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3904078b90 0x7f390407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:39.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.729+0000 7f391b7fe640 1 -- 172.21.15.37:0/4107058678 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5810+0+0 (secure 0 0 0) 0x7f390c09a060 con 0x7f39240b3430 2024-09-17T04:34:39.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.729+0000 7f3922d9d640 1 --2- 172.21.15.37:0/4107058678 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3904078b90 0x7f390407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:39.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.730+0000 7f3922d9d640 1 --2- 172.21.15.37:0/4107058678 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3904078b90 0x7f390407b0a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3914033000 tx=0x7f3914004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:39.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.733+0000 7f391b7fe640 1 -- 172.21.15.37:0/4107058678 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f39240d1850 con 0x7f39240b3430 2024-09-17T04:34:39.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.955+0000 7f392904d640 1 -- 172.21.15.37:0/4107058678 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f392406a150 con 0x7f39240b3430 2024-09-17T04:34:39.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.956+0000 7f391b7fe640 1 -- 172.21.15.37:0/4107058678 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5223 (secure 0 0 0) 0x7f390c062430 con 0x7f39240b3430 2024-09-17T04:34:39.956 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:39.956 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"btime":"2024-09-17T04:34:37:541562+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/199192833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":199192833},{"type":"v1","addr":"172.21.15.37:6833","nonce":199192833}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4832,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/4098913371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4098913371},{"type":"v1","addr":"172.21.15.92:6836","nonce":4098913371}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/383658044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":383658044},{"type":"v1","addr":"172.21.15.92:6837","nonce":383658044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:34:17.928780+0000","modified":"2024-09-17T04:34:28.244429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5208":{"gid":5208,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6835/4051839052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":4051839052},{"type":"v1","addr":"172.21.15.37:6835","nonce":4051839052}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":6,"enabled":true,"fs_name":"recovery_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5208,"qdb_cluster":[5208]},"id":3}]} 2024-09-17T04:34:39.956 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 29 2024-09-17T04:34:39.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.958+0000 7f392904d640 1 -- 172.21.15.37:0/4107058678 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3904078b90 msgr2=0x7f390407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:39.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.958+0000 7f392904d640 1 --2- 172.21.15.37:0/4107058678 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3904078b90 0x7f390407b0a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3914033000 tx=0x7f3914004730 comp rx=0 tx=0).stop 2024-09-17T04:34:39.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.958+0000 7f392904d640 1 -- 172.21.15.37:0/4107058678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39240b3430 msgr2=0x7f39241f67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:39.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.958+0000 7f392904d640 1 --2- 172.21.15.37:0/4107058678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39240b3430 0x7f39241f67e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f390c001eb0 tx=0x7f390c00c820 comp rx=0 tx=0).stop 2024-09-17T04:34:39.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.958+0000 7f392904d640 1 -- 172.21.15.37:0/4107058678 shutdown_connections 2024-09-17T04:34:39.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.958+0000 7f392904d640 1 --2- 172.21.15.37:0/4107058678 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3904078b90 0x7f390407b0a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:39.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.958+0000 7f392904d640 1 --2- 172.21.15.37:0/4107058678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39240d1440 0x7f392415f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:39.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.958+0000 7f392904d640 1 --2- 172.21.15.37:0/4107058678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39240b3430 0x7f39241f67e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:39.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.958+0000 7f392904d640 1 --2- 172.21.15.37:0/4107058678 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f392406f9f0 0x7f3924157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:39.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.958+0000 7f392904d640 1 -- 172.21.15.37:0/4107058678 >> 172.21.15.37:0/4107058678 conn(0x7f392408bef0 msgr2=0x7f3924160980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:39.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.958+0000 7f392904d640 1 -- 172.21.15.37:0/4107058678 shutdown_connections 2024-09-17T04:34:39.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:39.958+0000 7f392904d640 1 -- 172.21.15.37:0/4107058678 wait complete. 2024-09-17T04:34:39.969 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:34:40.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.086+0000 7f39a5d13640 1 -- 172.21.15.37:0/2853406302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3988004800 msgr2=0x7f3988004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:40.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.086+0000 7f39a5d13640 1 --2- 172.21.15.37:0/2853406302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3988004800 0x7f3988004c10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3994009b10 tx=0x7f3988005830 comp rx=0 tx=0).stop 2024-09-17T04:34:40.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.086+0000 7f399ffff640 1 -- 172.21.15.37:0/2853406302 reap_dead start 2024-09-17T04:34:40.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.086+0000 7f39a5d13640 1 -- 172.21.15.37:0/2853406302 shutdown_connections 2024-09-17T04:34:40.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.086+0000 7f39a5d13640 1 -- 172.21.15.37:0/2853406302 >> 172.21.15.37:0/2853406302 conn(0x7f39a008bef0 msgr2=0x7f39a00d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:40.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.086+0000 7f39a5d13640 1 -- 172.21.15.37:0/2853406302 shutdown_connections 2024-09-17T04:34:40.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.086+0000 7f39a5d13640 1 -- 172.21.15.37:0/2853406302 wait complete. 2024-09-17T04:34:40.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.088+0000 7f39a5d13640 1 Processor -- start 2024-09-17T04:34:40.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.088+0000 7f39a5d13640 1 -- start start 2024-09-17T04:34:40.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.090+0000 7f39a5d13640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3988004800 0x7f39a01f9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:40.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.090+0000 7f39a5d13640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39a00a0060 0x7f39a00bb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:40.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.090+0000 7f39a5d13640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f39a00cde90 0x7f39a00b8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:40.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.090+0000 7f39a5d13640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f39a00d4e50 con 0x7f39a00a0060 2024-09-17T04:34:40.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.090+0000 7f39a5d13640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f39a00b3b90 con 0x7f3988004800 2024-09-17T04:34:40.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.090+0000 7f39a5d13640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f39a00d33c0 con 0x7f39a00cde90 2024-09-17T04:34:40.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.090+0000 7f399effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39a00a0060 0x7f39a00bb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:40.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.090+0000 7f399effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39a00a0060 0x7f39a00bb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33202/0 (socket says 172.21.15.37:33202) 2024-09-17T04:34:40.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.090+0000 7f399effd640 1 -- 172.21.15.37:0/1427727968 learned_addr learned my addr 172.21.15.37:0/1427727968 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:40.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.090+0000 7f399ffff640 1 --2- 172.21.15.37:0/1427727968 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f39a00cde90 0x7f39a00b8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:40.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.090+0000 7f399f7fe640 1 --2- 172.21.15.37:0/1427727968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3988004800 0x7f39a01f9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:40.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.090+0000 7f399effd640 1 -- 172.21.15.37:0/1427727968 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f39a00cde90 msgr2=0x7f39a00b8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:40.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.090+0000 7f399effd640 1 --2- 172.21.15.37:0/1427727968 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f39a00cde90 0x7f39a00b8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:40.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.090+0000 7f399effd640 1 -- 172.21.15.37:0/1427727968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3988004800 msgr2=0x7f39a01f9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:40.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.090+0000 7f399effd640 1 --2- 172.21.15.37:0/1427727968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3988004800 0x7f39a01f9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:40.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.090+0000 7f399effd640 1 -- 172.21.15.37:0/1427727968 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a009f2e0 con 0x7f39a00a0060 2024-09-17T04:34:40.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.090+0000 7f399f7fe640 1 --2- 172.21.15.37:0/1427727968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3988004800 0x7f39a01f9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:40.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.090+0000 7f399effd640 1 --2- 172.21.15.37:0/1427727968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39a00a0060 0x7f39a00bb7f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3994009a60 tx=0x7f39940336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:40.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.091+0000 7f3987fff640 1 -- 172.21.15.37:0/1427727968 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3994038040 con 0x7f39a00a0060 2024-09-17T04:34:40.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.091+0000 7f3987fff640 1 -- 172.21.15.37:0/1427727968 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f399402b7b0 con 0x7f39a00a0060 2024-09-17T04:34:40.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.091+0000 7f39a5d13640 1 -- 172.21.15.37:0/1427727968 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a00bbf90 con 0x7f39a00a0060 2024-09-17T04:34:40.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.091+0000 7f3987fff640 1 -- 172.21.15.37:0/1427727968 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3994004280 con 0x7f39a00a0060 2024-09-17T04:34:40.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.091+0000 7f39a5d13640 1 -- 172.21.15.37:0/1427727968 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a009f860 con 0x7f39a00a0060 2024-09-17T04:34:40.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.092+0000 7f39a5d13640 1 -- 172.21.15.37:0/1427727968 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a0163370 con 0x7f39a00a0060 2024-09-17T04:34:40.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.092+0000 7f3987fff640 1 -- 172.21.15.37:0/1427727968 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3994045020 con 0x7f39a00a0060 2024-09-17T04:34:40.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.093+0000 7f3987fff640 1 --2- 172.21.15.37:0/1427727968 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f398c078b90 0x7f398c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:40.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.093+0000 7f3987fff640 1 -- 172.21.15.37:0/1427727968 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5810+0+0 (secure 0 0 0) 0x7f39940bb800 con 0x7f39a00a0060 2024-09-17T04:34:40.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.093+0000 7f399f7fe640 1 --2- 172.21.15.37:0/1427727968 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f398c078b90 0x7f398c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:40.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.094+0000 7f399f7fe640 1 --2- 172.21.15.37:0/1427727968 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f398c078b90 0x7f398c07b0a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3990005eb0 tx=0x7f3990005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:40.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.097+0000 7f3987fff640 1 -- 172.21.15.37:0/1427727968 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3994083610 con 0x7f39a00a0060 2024-09-17T04:34:40.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.321+0000 7f39a5d13640 1 -- 172.21.15.37:0/1427727968 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39a006a150 con 0x7f39a00a0060 2024-09-17T04:34:40.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.321+0000 7f3987fff640 1 -- 172.21.15.37:0/1427727968 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5223 (secure 0 0 0) 0x7f3994088040 con 0x7f39a00a0060 2024-09-17T04:34:40.321 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:40.321 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"btime":"2024-09-17T04:34:37:541562+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/199192833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":199192833},{"type":"v1","addr":"172.21.15.37:6833","nonce":199192833}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4832,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/4098913371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4098913371},{"type":"v1","addr":"172.21.15.92:6836","nonce":4098913371}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/383658044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":383658044},{"type":"v1","addr":"172.21.15.92:6837","nonce":383658044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T04:34:17.928780+0000","modified":"2024-09-17T04:34:28.244429+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5208":{"gid":5208,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6835/4051839052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":4051839052},{"type":"v1","addr":"172.21.15.37:6835","nonce":4051839052}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":6,"enabled":true,"fs_name":"recovery_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5208,"qdb_cluster":[5208]},"id":3}]} 2024-09-17T04:34:40.322 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 29 2024-09-17T04:34:40.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.323+0000 7f39a5d13640 1 -- 172.21.15.37:0/1427727968 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f398c078b90 msgr2=0x7f398c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:40.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.324+0000 7f39a5d13640 1 --2- 172.21.15.37:0/1427727968 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f398c078b90 0x7f398c07b0a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3990005eb0 tx=0x7f3990005f90 comp rx=0 tx=0).stop 2024-09-17T04:34:40.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.324+0000 7f39a5d13640 1 -- 172.21.15.37:0/1427727968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39a00a0060 msgr2=0x7f39a00bb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:40.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.324+0000 7f39a5d13640 1 --2- 172.21.15.37:0/1427727968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39a00a0060 0x7f39a00bb7f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3994009a60 tx=0x7f39940336f0 comp rx=0 tx=0).stop 2024-09-17T04:34:40.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.324+0000 7f39a5d13640 1 -- 172.21.15.37:0/1427727968 shutdown_connections 2024-09-17T04:34:40.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.324+0000 7f39a5d13640 1 --2- 172.21.15.37:0/1427727968 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f398c078b90 0x7f398c07b0a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:40.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.324+0000 7f39a5d13640 1 --2- 172.21.15.37:0/1427727968 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f39a00cde90 0x7f39a00b8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:40.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.324+0000 7f39a5d13640 1 --2- 172.21.15.37:0/1427727968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39a00a0060 0x7f39a00bb7f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:40.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.324+0000 7f39a5d13640 1 --2- 172.21.15.37:0/1427727968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3988004800 0x7f39a01f9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:40.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.324+0000 7f39a5d13640 1 -- 172.21.15.37:0/1427727968 >> 172.21.15.37:0/1427727968 conn(0x7f39a008bef0 msgr2=0x7f39a00d0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:40.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.324+0000 7f39a5d13640 1 -- 172.21.15.37:0/1427727968 shutdown_connections 2024-09-17T04:34:40.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.324+0000 7f39a5d13640 1 -- 172.21.15.37:0/1427727968 wait complete. 2024-09-17T04:34:40.335 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:34:40.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.451+0000 7f9dceed3640 1 -- 172.21.15.37:0/120723630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dc80b3430 msgr2=0x7f9dc80d2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:40.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.451+0000 7f9dceed3640 1 --2- 172.21.15.37:0/120723630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dc80b3430 0x7f9dc80d2ac0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0005fd0 tx=0x7f9dc002aa90 comp rx=0 tx=0).stop 2024-09-17T04:34:40.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.451+0000 7f9dceed3640 1 -- 172.21.15.37:0/120723630 shutdown_connections 2024-09-17T04:34:40.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.451+0000 7f9dceed3640 1 --2- 172.21.15.37:0/120723630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dc80b3430 0x7f9dc80d2ac0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:40.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.451+0000 7f9dceed3640 1 -- 172.21.15.37:0/120723630 >> 172.21.15.37:0/120723630 conn(0x7f9dc808bef0 msgr2=0x7f9dc80d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:40.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.451+0000 7f9dceed3640 1 -- 172.21.15.37:0/120723630 shutdown_connections 2024-09-17T04:34:40.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.451+0000 7f9dceed3640 1 -- 172.21.15.37:0/120723630 wait complete. 2024-09-17T04:34:40.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.453+0000 7f9dceed3640 1 Processor -- start 2024-09-17T04:34:40.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.453+0000 7f9dceed3640 1 -- start start 2024-09-17T04:34:40.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.455+0000 7f9dceed3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dc806f9f0 0x7f9dc81578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:40.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.455+0000 7f9dceed3640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9dc80b3430 0x7f9dc815a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:40.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.455+0000 7f9dceed3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dc815a850 0x7f9dc815ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:40.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.455+0000 7f9dceed3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc80d4e50 con 0x7f9dc806f9f0 2024-09-17T04:34:40.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.455+0000 7f9dceed3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc80b3b90 con 0x7f9dc815a850 2024-09-17T04:34:40.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.455+0000 7f9dceed3640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f9dc80d3000 con 0x7f9dc80b3430 2024-09-17T04:34:40.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.455+0000 7f9dccc6f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dc806f9f0 0x7f9dc81578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:40.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.455+0000 7f9dccc6f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dc806f9f0 0x7f9dc81578b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33212/0 (socket says 172.21.15.37:33212) 2024-09-17T04:34:40.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.455+0000 7f9dccc6f640 1 -- 172.21.15.37:0/3884952646 learned_addr learned my addr 172.21.15.37:0/3884952646 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.455+0000 7f9dc7fff640 1 --2- 172.21.15.37:0/3884952646 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9dc80b3430 0x7f9dc815a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.455+0000 7f9dcd470640 1 --2- 172.21.15.37:0/3884952646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dc815a850 0x7f9dc815ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.455+0000 7f9dccc6f640 1 -- 172.21.15.37:0/3884952646 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9dc80b3430 msgr2=0x7f9dc815a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.455+0000 7f9dccc6f640 1 --2- 172.21.15.37:0/3884952646 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9dc80b3430 0x7f9dc815a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.455+0000 7f9dccc6f640 1 -- 172.21.15.37:0/3884952646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dc815a850 msgr2=0x7f9dc815ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.455+0000 7f9dccc6f640 1 --2- 172.21.15.37:0/3884952646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dc815a850 0x7f9dc815ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.455+0000 7f9dccc6f640 1 -- 172.21.15.37:0/3884952646 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc80b9240 con 0x7f9dc806f9f0 2024-09-17T04:34:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.455+0000 7f9dccc6f640 1 --2- 172.21.15.37:0/3884952646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dc806f9f0 0x7f9dc81578b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc00c3c0 tx=0x7f9dbc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.455+0000 7f9dc57fa640 1 -- 172.21.15.37:0/3884952646 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc017070 con 0x7f9dc806f9f0 2024-09-17T04:34:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.455+0000 7f9dc57fa640 1 -- 172.21.15.37:0/3884952646 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9dbc0053c0 con 0x7f9dc806f9f0 2024-09-17T04:34:40.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.455+0000 7f9dceed3640 1 -- 172.21.15.37:0/3884952646 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc80b6ab0 con 0x7f9dc806f9f0 2024-09-17T04:34:40.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.456+0000 7f9dc57fa640 1 -- 172.21.15.37:0/3884952646 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc003850 con 0x7f9dc806f9f0 2024-09-17T04:34:40.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.456+0000 7f9dceed3640 1 -- 172.21.15.37:0/3884952646 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc80b8760 con 0x7f9dc806f9f0 2024-09-17T04:34:40.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.456+0000 7f9dceed3640 1 -- 172.21.15.37:0/3884952646 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dc80d34d0 con 0x7f9dc806f9f0 2024-09-17T04:34:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.457+0000 7f9dc57fa640 1 -- 172.21.15.37:0/3884952646 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9dbc01a070 con 0x7f9dc806f9f0 2024-09-17T04:34:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.458+0000 7f9dc57fa640 1 --2- 172.21.15.37:0/3884952646 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9dac078b90 0x7f9dac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.458+0000 7f9dc57fa640 1 -- 172.21.15.37:0/3884952646 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5810+0+0 (secure 0 0 0) 0x7f9dbc09a150 con 0x7f9dc806f9f0 2024-09-17T04:34:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.458+0000 7f9dc7fff640 1 --2- 172.21.15.37:0/3884952646 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9dac078b90 0x7f9dac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.459+0000 7f9dc7fff640 1 --2- 172.21.15.37:0/3884952646 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9dac078b90 0x7f9dac07b0a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80b49b0 tx=0x7f9db4009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:40.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.463+0000 7f9dc57fa640 1 -- 172.21.15.37:0/3884952646 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9dbc061e40 con 0x7f9dc806f9f0 2024-09-17T04:34:40.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.649+0000 7f9dceed3640 1 -- 172.21.15.37:0/3884952646 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9dc806a150 con 0x7f9dc806f9f0 2024-09-17T04:34:40.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.650+0000 7f9dc57fa640 1 -- 172.21.15.37:0/3884952646 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16813 (secure 0 0 0) 0x7f9dbc066870 con 0x7f9dc806f9f0 2024-09-17T04:34:40.650 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:40.650 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":42,"fsid":"a908a603-430c-4209-8294-0c3460b35f42","created":"2024-09-17T04:32:54.139221+0000","modified":"2024-09-17T04:34:38.547869+0000","last_up_change":"2024-09-17T04:33:06.922210+0000","last_in_change":"2024-09-17T04:32:57.754118+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:33:08.461907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T04:33:58.446650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"recovery_fs_metadata","create_time":"2024-09-17T04:34:16.436885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ea4b8d1-0d22-4dd9-b913-9df585ef24ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6809","nonce":2283587593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6811","nonce":2283587593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6815","nonce":2283587593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6813","nonce":2283587593}]},"public_addr":"172.21.15.37:6809/2283587593","cluster_addr":"172.21.15.37:6811/2283587593","heartbeat_back_addr":"172.21.15.37:6815/2283587593","heartbeat_front_addr":"172.21.15.37:6813/2283587593","state":["exists","up"]},{"osd":1,"uuid":"ac745a33-e8ad-4083-9c18-71bf81982b76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6825","nonce":2657955935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6827","nonce":2657955935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6831","nonce":2657955935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6829","nonce":2657955935}]},"public_addr":"172.21.15.37:6825/2657955935","cluster_addr":"172.21.15.37:6827/2657955935","heartbeat_back_addr":"172.21.15.37:6831/2657955935","heartbeat_front_addr":"172.21.15.37:6829/2657955935","state":["exists","up"]},{"osd":2,"uuid":"3c63f3d8-132a-4735-b0bc-d64313a46503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6817","nonce":579720176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6819","nonce":579720176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6823","nonce":579720176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6821","nonce":579720176}]},"public_addr":"172.21.15.37:6817/579720176","cluster_addr":"172.21.15.37:6819/579720176","heartbeat_back_addr":"172.21.15.37:6823/579720176","heartbeat_front_addr":"172.21.15.37:6821/579720176","state":["exists","up"]},{"osd":3,"uuid":"4f8c6ca2-ea1d-4610-9127-52c4dd8bc8e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6801","nonce":809975826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6803","nonce":809975826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6807","nonce":809975826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6805","nonce":809975826}]},"public_addr":"172.21.15.37:6801/809975826","cluster_addr":"172.21.15.37:6803/809975826","heartbeat_back_addr":"172.21.15.37:6807/809975826","heartbeat_front_addr":"172.21.15.37:6805/809975826","state":["exists","up"]},{"osd":4,"uuid":"63d0c4ab-f115-47b4-9611-12a10153473a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6819","nonce":2509785752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6821","nonce":2509785752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6825","nonce":2509785752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6823","nonce":2509785752}]},"public_addr":"172.21.15.92:6819/2509785752","cluster_addr":"172.21.15.92:6821/2509785752","heartbeat_back_addr":"172.21.15.92:6825/2509785752","heartbeat_front_addr":"172.21.15.92:6823/2509785752","state":["exists","up"]},{"osd":5,"uuid":"4c56c04d-0c07-4f76-9d75-6335121310ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6827","nonce":248263832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6829","nonce":248263832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6833","nonce":248263832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6831","nonce":248263832}]},"public_addr":"172.21.15.92:6827/248263832","cluster_addr":"172.21.15.92:6829/248263832","heartbeat_back_addr":"172.21.15.92:6833/248263832","heartbeat_front_addr":"172.21.15.92:6831/248263832","state":["exists","up"]},{"osd":6,"uuid":"b30eae7c-096a-4d24-ac51-96b8b13ecaee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6811","nonce":4078365985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6813","nonce":4078365985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6817","nonce":4078365985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6815","nonce":4078365985}]},"public_addr":"172.21.15.92:6811/4078365985","cluster_addr":"172.21.15.92:6813/4078365985","heartbeat_back_addr":"172.21.15.92:6817/4078365985","heartbeat_front_addr":"172.21.15.92:6815/4078365985","state":["exists","up"]},{"osd":7,"uuid":"62f12540-d374-4a4c-9ff7-63b7304fe77f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6803","nonce":3444787223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6805","nonce":3444787223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6809","nonce":3444787223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6807","nonce":3444787223}]},"public_addr":"172.21.15.92:6803/3444787223","cluster_addr":"172.21.15.92:6805/3444787223","heartbeat_back_addr":"172.21.15.92:6809/3444787223","heartbeat_front_addr":"172.21.15.92:6807/3444787223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.006420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.111135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.103325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.072656+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.200989+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.060455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.774773+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.865778+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/1358600605":"2024-09-18T04:34:15.699884+0000","172.21.15.37:6833/1358600605":"2024-09-18T04:34:15.699884+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:34:40.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.652+0000 7f9dceed3640 1 -- 172.21.15.37:0/3884952646 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9dac078b90 msgr2=0x7f9dac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:40.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.652+0000 7f9dceed3640 1 --2- 172.21.15.37:0/3884952646 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9dac078b90 0x7f9dac07b0a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80b49b0 tx=0x7f9db4009060 comp rx=0 tx=0).stop 2024-09-17T04:34:40.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.652+0000 7f9dceed3640 1 -- 172.21.15.37:0/3884952646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dc806f9f0 msgr2=0x7f9dc81578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:40.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.652+0000 7f9dceed3640 1 --2- 172.21.15.37:0/3884952646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dc806f9f0 0x7f9dc81578b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc00c3c0 tx=0x7f9dbc00c880 comp rx=0 tx=0).stop 2024-09-17T04:34:40.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.652+0000 7f9dceed3640 1 -- 172.21.15.37:0/3884952646 shutdown_connections 2024-09-17T04:34:40.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.652+0000 7f9dceed3640 1 --2- 172.21.15.37:0/3884952646 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9dac078b90 0x7f9dac07b0a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:40.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.652+0000 7f9dceed3640 1 --2- 172.21.15.37:0/3884952646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dc815a850 0x7f9dc815ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:40.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.652+0000 7f9dceed3640 1 --2- 172.21.15.37:0/3884952646 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9dc80b3430 0x7f9dc815a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:40.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.652+0000 7f9dceed3640 1 --2- 172.21.15.37:0/3884952646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dc806f9f0 0x7f9dc81578b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:40.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.652+0000 7f9dceed3640 1 -- 172.21.15.37:0/3884952646 >> 172.21.15.37:0/3884952646 conn(0x7f9dc808bef0 msgr2=0x7f9dc8161940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:40.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.652+0000 7f9dceed3640 1 -- 172.21.15.37:0/3884952646 shutdown_connections 2024-09-17T04:34:40.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.652+0000 7f9dceed3640 1 -- 172.21.15.37:0/3884952646 wait complete. 2024-09-17T04:34:40.664 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail recovery_fs 2024-09-17T04:34:40.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.781+0000 7f130e0de640 1 -- 172.21.15.37:0/230801004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13080cd1b0 msgr2=0x7f13080cd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:40.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.781+0000 7f130e0de640 1 --2- 172.21.15.37:0/230801004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13080cd1b0 0x7f13080cd5c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1308074c60 tx=0x7f130002a8b0 comp rx=0 tx=0).stop 2024-09-17T04:34:40.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.781+0000 7f130e0de640 1 -- 172.21.15.37:0/230801004 shutdown_connections 2024-09-17T04:34:40.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.781+0000 7f130e0de640 1 --2- 172.21.15.37:0/230801004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13080cd1b0 0x7f13080cd5c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:40.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.781+0000 7f130e0de640 1 --2- 172.21.15.37:0/230801004 >> v2:172.21.15.92:3300/0 conn(0x7f1308072a90 0x7f13080cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:40.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.781+0000 7f130e0de640 1 -- 172.21.15.37:0/230801004 >> 172.21.15.37:0/230801004 conn(0x7f13080d2460 msgr2=0x7f13080d2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:40.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.781+0000 7f130e0de640 1 -- 172.21.15.37:0/230801004 shutdown_connections 2024-09-17T04:34:40.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.781+0000 7f130e0de640 1 -- 172.21.15.37:0/230801004 wait complete. 2024-09-17T04:34:40.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.783+0000 7f130e0de640 1 Processor -- start 2024-09-17T04:34:40.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.783+0000 7f130e0de640 1 -- start start 2024-09-17T04:34:40.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.784+0000 7f130e0de640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1308072a90 0x7f1308159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:40.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.784+0000 7f130e0de640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f13080cd1b0 0x7f13081565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:40.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.784+0000 7f130e0de640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13080cf590 0x7f130815f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:40.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.784+0000 7f130e0de640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f13080cdc80 con 0x7f13080cf590 2024-09-17T04:34:40.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.784+0000 7f130e0de640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f13080cdb00 con 0x7f1308072a90 2024-09-17T04:34:40.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.784+0000 7f130e0de640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f1308161100 con 0x7f13080cd1b0 2024-09-17T04:34:40.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.784+0000 7f1307fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13080cf590 0x7f130815f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:40.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.784+0000 7f1307fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13080cf590 0x7f130815f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33222/0 (socket says 172.21.15.37:33222) 2024-09-17T04:34:40.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.784+0000 7f1307fff640 1 -- 172.21.15.37:0/1069725785 learned_addr learned my addr 172.21.15.37:0/1069725785 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:40.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.785+0000 7f13077fe640 1 --2- 172.21.15.37:0/1069725785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1308072a90 0x7f1308159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:40.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.785+0000 7f1306ffd640 1 --2- 172.21.15.37:0/1069725785 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f13080cd1b0 0x7f13081565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:40.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.785+0000 7f1307fff640 1 -- 172.21.15.37:0/1069725785 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f13080cd1b0 msgr2=0x7f13081565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:40.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.785+0000 7f1307fff640 1 --2- 172.21.15.37:0/1069725785 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f13080cd1b0 0x7f13081565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:40.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.785+0000 7f1307fff640 1 -- 172.21.15.37:0/1069725785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1308072a90 msgr2=0x7f1308159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:40.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.785+0000 7f1307fff640 1 --2- 172.21.15.37:0/1069725785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1308072a90 0x7f1308159e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:40.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.785+0000 7f1307fff640 1 -- 172.21.15.37:0/1069725785 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13081f3eb0 con 0x7f13080cf590 2024-09-17T04:34:40.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.785+0000 7f1307fff640 1 --2- 172.21.15.37:0/1069725785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13080cf590 0x7f130815f7a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f130002aea0 tx=0x7f13000039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:40.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.785+0000 7f12effff640 1 -- 172.21.15.37:0/1069725785 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1300031990 con 0x7f13080cf590 2024-09-17T04:34:40.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.785+0000 7f12effff640 1 -- 172.21.15.37:0/1069725785 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1300031c20 con 0x7f13080cf590 2024-09-17T04:34:40.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.785+0000 7f130e0de640 1 -- 172.21.15.37:0/1069725785 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13081f4190 con 0x7f13080cf590 2024-09-17T04:34:40.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.785+0000 7f12effff640 1 -- 172.21.15.37:0/1069725785 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1300033330 con 0x7f13080cf590 2024-09-17T04:34:40.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.785+0000 7f130e0de640 1 -- 172.21.15.37:0/1069725785 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13081f4750 con 0x7f13080cf590 2024-09-17T04:34:40.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.786+0000 7f130e0de640 1 -- 172.21.15.37:0/1069725785 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13080cf050 con 0x7f13080cf590 2024-09-17T04:34:40.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.787+0000 7f12effff640 1 -- 172.21.15.37:0/1069725785 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f13000334d0 con 0x7f13080cf590 2024-09-17T04:34:40.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.788+0000 7f12effff640 1 --2- 172.21.15.37:0/1069725785 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f12f4078b90 0x7f12f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:40.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.788+0000 7f12effff640 1 -- 172.21.15.37:0/1069725785 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5810+0+0 (secure 0 0 0) 0x7f13000bc1d0 con 0x7f13080cf590 2024-09-17T04:34:40.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.788+0000 7f13077fe640 1 --2- 172.21.15.37:0/1069725785 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f12f4078b90 0x7f12f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:40.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.788+0000 7f13077fe640 1 --2- 172.21.15.37:0/1069725785 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f12f4078b90 0x7f12f407b0a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f12f8004610 tx=0x7f12f800b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:40.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:40.792+0000 7f12effff640 1 -- 172.21.15.37:0/1069725785 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f130003f030 con 0x7f13080cf590 2024-09-17T04:34:41.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.016+0000 7f130e0de640 1 -- 172.21.15.37:0/1069725785 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "recovery_fs"} v 0) -- 0x7f130806a150 con 0x7f13080cf590 2024-09-17T04:34:41.016 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-09-17T04:34:41.017+0000 7f4732e82640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T04:34:41.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.020+0000 7f12effff640 1 -- 172.21.15.37:0/1069725785 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "recovery_fs"}]=0 recovery_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) ==== 171+0+0 (secure 0 0 0) 0x7f13000317b0 con 0x7f13080cf590 2024-09-17T04:34:41.020 INFO:teuthology.orchestra.run.smithi037.stderr:recovery_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T04:34:41.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.022+0000 7f130e0de640 1 -- 172.21.15.37:0/1069725785 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f12f4078b90 msgr2=0x7f12f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:41.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.022+0000 7f130e0de640 1 --2- 172.21.15.37:0/1069725785 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f12f4078b90 0x7f12f407b0a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f12f8004610 tx=0x7f12f800b2e0 comp rx=0 tx=0).stop 2024-09-17T04:34:41.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.022+0000 7f130e0de640 1 -- 172.21.15.37:0/1069725785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13080cf590 msgr2=0x7f130815f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:41.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.022+0000 7f130e0de640 1 --2- 172.21.15.37:0/1069725785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13080cf590 0x7f130815f7a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f130002aea0 tx=0x7f13000039f0 comp rx=0 tx=0).stop 2024-09-17T04:34:41.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.022+0000 7f130e0de640 1 -- 172.21.15.37:0/1069725785 shutdown_connections 2024-09-17T04:34:41.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.022+0000 7f130e0de640 1 --2- 172.21.15.37:0/1069725785 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f12f4078b90 0x7f12f407b0a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:41.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.022+0000 7f130e0de640 1 --2- 172.21.15.37:0/1069725785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13080cf590 0x7f130815f7a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:41.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.022+0000 7f130e0de640 1 --2- 172.21.15.37:0/1069725785 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f13080cd1b0 0x7f13081565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:41.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.022+0000 7f130e0de640 1 --2- 172.21.15.37:0/1069725785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1308072a90 0x7f1308159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:41.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.022+0000 7f130e0de640 1 -- 172.21.15.37:0/1069725785 >> 172.21.15.37:0/1069725785 conn(0x7f13080d2460 msgr2=0x7f13080b6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:41.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.022+0000 7f130e0de640 1 -- 172.21.15.37:0/1069725785 shutdown_connections 2024-09-17T04:34:41.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.022+0000 7f130e0de640 1 -- 172.21.15.37:0/1069725785 wait complete. 2024-09-17T04:34:41.026 INFO:tasks.ceph.mds.a.smithi037.stderr: -3765> 2024-09-17T04:34:25.240+0000 7f55101d1640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2024-09-17T04:34:41.027 INFO:tasks.ceph.mds.a.smithi037.stderr: -1997> 2024-09-17T04:34:30.318+0000 7f55101d1640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2024-09-17T04:34:41.027 INFO:tasks.ceph.mds.a.smithi037.stderr: -1845> 2024-09-17T04:34:30.319+0000 7f55101d1640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55b7e6409180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2024-09-17T04:34:41.027 INFO:tasks.ceph.mds.a.smithi037.stderr: -1713> 2024-09-17T04:34:30.320+0000 7f55101d1640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v1 ap=1 DIRTYPARENT s=6291456 n() | request=0 lock=0 dirtyparent=1 authpin=1 scrubqueue=0 0x55b7e6690c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(5)0x10000000001:[<0x10000000000/sixmegs v8>,<0x1/subdir v10>]//[]","memoryvalue":"(5)0x10000000001:[<0x10000000000/sixmegs v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-17T04:34:41.027 INFO:tasks.ceph.mds.a.smithi037.stderr: -1335> 2024-09-17T04:34:30.324+0000 7f55101d1640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v4 pv6 ap=5 snaprealm=0x55b7e66066c0 DIRTYPARENT f(v1 1=0+1) n(v0 1=0+1)/n() (inest mix->lock(2) w=1 flushing) (ifile mix->sync w=1 flushing) | dirtyscattered=2 lock=2 dirfrag=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x55b7e6612580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2024-09-17T04:34:41.027 INFO:tasks.ceph.mds.a.smithi037.stderr: -1188> 2024-09-17T04:34:31.710+0000 7f55141d9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T04:34:41.035 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm recovery_fs --yes-i-really-mean-it 2024-09-17T04:34:41.082 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-09-17T04:34:41.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.158+0000 7f50c8b07640 1 -- 172.21.15.37:0/2168827212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50c40ccf90 msgr2=0x7f50c40cd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:41.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.158+0000 7f50c8b07640 1 --2- 172.21.15.37:0/2168827212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50c40ccf90 0x7f50c40cd3a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f50b8005fd0 tx=0x7f50b802a5d0 comp rx=0 tx=0).stop 2024-09-17T04:34:41.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.158+0000 7f50c8b07640 1 -- 172.21.15.37:0/2168827212 shutdown_connections 2024-09-17T04:34:41.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.158+0000 7f50c8b07640 1 --2- 172.21.15.37:0/2168827212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50c40ccf90 0x7f50c40cd3a0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:41.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.158+0000 7f50c8b07640 1 --2- 172.21.15.37:0/2168827212 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f50c40d2fa0 0x7f50c40d0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:41.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.158+0000 7f50c8b07640 1 -- 172.21.15.37:0/2168827212 >> 172.21.15.37:0/2168827212 conn(0x7f50c40d2480 msgr2=0x7f50c40d2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:41.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.158+0000 7f50c8b07640 1 -- 172.21.15.37:0/2168827212 shutdown_connections 2024-09-17T04:34:41.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.158+0000 7f50c8b07640 1 -- 172.21.15.37:0/2168827212 wait complete. 2024-09-17T04:34:41.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.160+0000 7f50c8b07640 1 Processor -- start 2024-09-17T04:34:41.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.160+0000 7f50c8b07640 1 -- start start 2024-09-17T04:34:41.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.161+0000 7f50c8b07640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f50c4071210 0x7f50c41f6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:41.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.161+0000 7f50c8b07640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50c40ccf90 0x7f50c415d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:41.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.161+0000 7f50c8b07640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50c40d2fa0 0x7f50c406ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:41.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.161+0000 7f50c8b07640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f50c40cda70 con 0x7f50c40ccf90 2024-09-17T04:34:41.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.161+0000 7f50c8b07640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f50c40b8010 con 0x7f50c40d2fa0 2024-09-17T04:34:41.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.161+0000 7f50c8b07640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f50c40d11b0 con 0x7f50c4071210 2024-09-17T04:34:41.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.161+0000 7f50c1d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50c40ccf90 0x7f50c415d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:41.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.161+0000 7f50c1d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50c40ccf90 0x7f50c415d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33236/0 (socket says 172.21.15.37:33236) 2024-09-17T04:34:41.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.161+0000 7f50c1d9b640 1 -- 172.21.15.37:0/4219748769 learned_addr learned my addr 172.21.15.37:0/4219748769 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:41.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.162+0000 7f50c259c640 1 --2- 172.21.15.37:0/4219748769 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f50c4071210 0x7f50c41f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:41.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.162+0000 7f50c2d9d640 1 --2- 172.21.15.37:0/4219748769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50c40d2fa0 0x7f50c406ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:41.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.162+0000 7f50c1d9b640 1 -- 172.21.15.37:0/4219748769 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f50c4071210 msgr2=0x7f50c41f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:41.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.162+0000 7f50c1d9b640 1 --2- 172.21.15.37:0/4219748769 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f50c4071210 0x7f50c41f6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:41.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.162+0000 7f50c1d9b640 1 -- 172.21.15.37:0/4219748769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50c40d2fa0 msgr2=0x7f50c406ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:41.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.162+0000 7f50c1d9b640 1 --2- 172.21.15.37:0/4219748769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50c40d2fa0 0x7f50c406ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:41.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.162+0000 7f50c1d9b640 1 -- 172.21.15.37:0/4219748769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c41f6780 con 0x7f50c40ccf90 2024-09-17T04:34:41.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.162+0000 7f50c2d9d640 1 --2- 172.21.15.37:0/4219748769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50c40d2fa0 0x7f50c406ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:41.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.162+0000 7f50c1d9b640 1 --2- 172.21.15.37:0/4219748769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50c40ccf90 0x7f50c415d7f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f50b8006010 tx=0x7f50b80035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:41.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.162+0000 7f50b2ffd640 1 -- 172.21.15.37:0/4219748769 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b803e030 con 0x7f50c40ccf90 2024-09-17T04:34:41.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.162+0000 7f50b2ffd640 1 -- 172.21.15.37:0/4219748769 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50b8003ae0 con 0x7f50c40ccf90 2024-09-17T04:34:41.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.162+0000 7f50b2ffd640 1 -- 172.21.15.37:0/4219748769 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b8035040 con 0x7f50c40ccf90 2024-09-17T04:34:41.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.162+0000 7f50c8b07640 1 -- 172.21.15.37:0/4219748769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50c415dd30 con 0x7f50c40ccf90 2024-09-17T04:34:41.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.162+0000 7f50c8b07640 1 -- 172.21.15.37:0/4219748769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50c40cb820 con 0x7f50c40ccf90 2024-09-17T04:34:41.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.163+0000 7f50c8b07640 1 -- 172.21.15.37:0/4219748769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50c40cb560 con 0x7f50c40ccf90 2024-09-17T04:34:41.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.164+0000 7f50b2ffd640 1 -- 172.21.15.37:0/4219748769 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f50b8041030 con 0x7f50c40ccf90 2024-09-17T04:34:41.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.164+0000 7f50b2ffd640 1 --2- 172.21.15.37:0/4219748769 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f50a4078b90 0x7f50a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:41.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.164+0000 7f50b2ffd640 1 -- 172.21.15.37:0/4219748769 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5896+0+0 (secure 0 0 0) 0x7f50b80bb2a0 con 0x7f50c40ccf90 2024-09-17T04:34:41.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.165+0000 7f50c259c640 1 --2- 172.21.15.37:0/4219748769 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f50a4078b90 0x7f50a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:41.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.165+0000 7f50c259c640 1 --2- 172.21.15.37:0/4219748769 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f50a4078b90 0x7f50a407b0a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f50b4005fd0 tx=0x7f50b40094a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:41.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.169+0000 7f50b2ffd640 1 -- 172.21.15.37:0/4219748769 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f50c41f6780 con 0x7f50c40ccf90 2024-09-17T04:34:41.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:41.391+0000 7f50c8b07640 1 -- 172.21.15.37:0/4219748769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "recovery_fs", "yes_i_really_mean_it": true} v 0) -- 0x7f50c40cd3a0 con 0x7f50c40ccf90 2024-09-17T04:34:42.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.023+0000 7f50b2ffd640 1 -- 172.21.15.37:0/4219748769 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "recovery_fs", "yes_i_really_mean_it": true}]=0 v31) ==== 109+0+0 (secure 0 0 0) 0x7f50b8083740 con 0x7f50c40ccf90 2024-09-17T04:34:42.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.026+0000 7f50c8b07640 1 -- 172.21.15.37:0/4219748769 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f50a4078b90 msgr2=0x7f50a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:42.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.026+0000 7f50c8b07640 1 --2- 172.21.15.37:0/4219748769 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f50a4078b90 0x7f50a407b0a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f50b4005fd0 tx=0x7f50b40094a0 comp rx=0 tx=0).stop 2024-09-17T04:34:42.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.026+0000 7f50c8b07640 1 -- 172.21.15.37:0/4219748769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50c40ccf90 msgr2=0x7f50c415d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:42.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.026+0000 7f50c8b07640 1 --2- 172.21.15.37:0/4219748769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50c40ccf90 0x7f50c415d7f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f50b8006010 tx=0x7f50b80035f0 comp rx=0 tx=0).stop 2024-09-17T04:34:42.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.026+0000 7f50c8b07640 1 -- 172.21.15.37:0/4219748769 shutdown_connections 2024-09-17T04:34:42.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.026+0000 7f50c8b07640 1 --2- 172.21.15.37:0/4219748769 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f50a4078b90 0x7f50a407b0a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:42.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.026+0000 7f50c8b07640 1 --2- 172.21.15.37:0/4219748769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50c40d2fa0 0x7f50c406ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:42.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.026+0000 7f50c8b07640 1 --2- 172.21.15.37:0/4219748769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50c40ccf90 0x7f50c415d7f0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:42.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.026+0000 7f50c8b07640 1 --2- 172.21.15.37:0/4219748769 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f50c4071210 0x7f50c41f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:42.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.026+0000 7f50c8b07640 1 -- 172.21.15.37:0/4219748769 >> 172.21.15.37:0/4219748769 conn(0x7f50c40d2480 msgr2=0x7f50c41639a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:42.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.026+0000 7f50c8b07640 1 -- 172.21.15.37:0/4219748769 shutdown_connections 2024-09-17T04:34:42.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.026+0000 7f50c8b07640 1 -- 172.21.15.37:0/4219748769 wait complete. 2024-09-17T04:34:42.041 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm recovery_fs_metadata recovery_fs_metadata --yes-i-really-really-mean-it 2024-09-17T04:34:42.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.169+0000 7febbbb20640 1 -- 172.21.15.37:0/683322454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb40b6750 msgr2=0x7febb4161ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:42.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.169+0000 7febbbb20640 1 --2- 172.21.15.37:0/683322454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb40b6750 0x7febb4161ff0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7febb40b7110 tx=0x7feba402a5f0 comp rx=0 tx=0).stop 2024-09-17T04:34:42.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.169+0000 7febbbb20640 1 -- 172.21.15.37:0/683322454 shutdown_connections 2024-09-17T04:34:42.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.169+0000 7febbbb20640 1 --2- 172.21.15.37:0/683322454 >> v2:172.21.15.92:3300/0 conn(0x7febb40b3a10 0x7febb40d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:42.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.169+0000 7febbbb20640 1 --2- 172.21.15.37:0/683322454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb40b6750 0x7febb4161ff0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:42.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.169+0000 7febbbb20640 1 -- 172.21.15.37:0/683322454 >> 172.21.15.37:0/683322454 conn(0x7febb408bef0 msgr2=0x7febb40d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:42.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.169+0000 7febbbb20640 1 -- 172.21.15.37:0/683322454 shutdown_connections 2024-09-17T04:34:42.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.169+0000 7febbbb20640 1 -- 172.21.15.37:0/683322454 wait complete. 2024-09-17T04:34:42.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.171+0000 7febbbb20640 1 Processor -- start 2024-09-17T04:34:42.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.171+0000 7febbbb20640 1 -- start start 2024-09-17T04:34:42.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.172+0000 7febbbb20640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb40b3a10 0x7febb40c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:42.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.172+0000 7febbbb20640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7febb40c9d70 0x7febb40cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:42.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.172+0000 7febbbb20640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febb40c7ad0 0x7febb41f8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:42.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.172+0000 7febbbb20640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7febb40d2db0 con 0x7febb40b3a10 2024-09-17T04:34:42.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.172+0000 7febbbb20640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7febb40d2c30 con 0x7febb40c7ad0 2024-09-17T04:34:42.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.172+0000 7febbbb20640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7febb40d2f30 con 0x7febb40c9d70 2024-09-17T04:34:42.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.172+0000 7febb98bc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb40b3a10 0x7febb40c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:42.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.172+0000 7febb98bc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb40b3a10 0x7febb40c9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33254/0 (socket says 172.21.15.37:33254) 2024-09-17T04:34:42.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.172+0000 7febb98bc640 1 -- 172.21.15.37:0/2614180972 learned_addr learned my addr 172.21.15.37:0/2614180972 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:42.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.173+0000 7febba0bd640 1 --2- 172.21.15.37:0/2614180972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febb40c7ad0 0x7febb41f8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:42.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.173+0000 7febb90bb640 1 --2- 172.21.15.37:0/2614180972 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7febb40c9d70 0x7febb40cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:42.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.173+0000 7febb98bc640 1 -- 172.21.15.37:0/2614180972 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7febb40c9d70 msgr2=0x7febb40cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:42.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.173+0000 7febb98bc640 1 --2- 172.21.15.37:0/2614180972 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7febb40c9d70 0x7febb40cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:42.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.173+0000 7febb98bc640 1 -- 172.21.15.37:0/2614180972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febb40c7ad0 msgr2=0x7febb41f8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:42.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.173+0000 7febb98bc640 1 --2- 172.21.15.37:0/2614180972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febb40c7ad0 0x7febb41f8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:42.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.173+0000 7febb98bc640 1 -- 172.21.15.37:0/2614180972 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb40c9b50 con 0x7febb40b3a10 2024-09-17T04:34:42.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.173+0000 7febb90bb640 1 --2- 172.21.15.37:0/2614180972 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7febb40c9d70 0x7febb40cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:42.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.173+0000 7febba0bd640 1 --2- 172.21.15.37:0/2614180972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febb40c7ad0 0x7febb41f8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:42.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.173+0000 7febb98bc640 1 --2- 172.21.15.37:0/2614180972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb40b3a10 0x7febb40c9090 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7febb000c3c0 tx=0x7febb000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:42.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.173+0000 7febaa7fc640 1 -- 172.21.15.37:0/2614180972 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb0017070 con 0x7febb40b3a10 2024-09-17T04:34:42.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.173+0000 7febaa7fc640 1 -- 172.21.15.37:0/2614180972 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febb00035f0 con 0x7febb40b3a10 2024-09-17T04:34:42.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.173+0000 7febbbb20640 1 -- 172.21.15.37:0/2614180972 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febb40cc9a0 con 0x7febb40b3a10 2024-09-17T04:34:42.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.173+0000 7febaa7fc640 1 -- 172.21.15.37:0/2614180972 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb0003a40 con 0x7febb40b3a10 2024-09-17T04:34:42.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.173+0000 7febbbb20640 1 -- 172.21.15.37:0/2614180972 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febb40cb6a0 con 0x7febb40b3a10 2024-09-17T04:34:42.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.174+0000 7febbbb20640 1 -- 172.21.15.37:0/2614180972 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febb4069f30 con 0x7febb40b3a10 2024-09-17T04:34:42.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.175+0000 7febaa7fc640 1 -- 172.21.15.37:0/2614180972 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7febb001a070 con 0x7febb40b3a10 2024-09-17T04:34:42.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.176+0000 7febaa7fc640 1 --2- 172.21.15.37:0/2614180972 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7feb98078bc0 0x7feb9807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:42.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.176+0000 7febaa7fc640 1 -- 172.21.15.37:0/2614180972 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5896+0+0 (secure 0 0 0) 0x7febb009a140 con 0x7febb40b3a10 2024-09-17T04:34:42.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.176+0000 7febb90bb640 1 --2- 172.21.15.37:0/2614180972 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7feb98078bc0 0x7feb9807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:42.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.177+0000 7febb90bb640 1 --2- 172.21.15.37:0/2614180972 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7feb98078bc0 0x7feb9807b0d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7feba4034000 tx=0x7feba4004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:42.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.181+0000 7febaa7fc640 1 -- 172.21.15.37:0/2614180972 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7febb4069f30 con 0x7febb40b3a10 2024-09-17T04:34:42.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:42.372+0000 7febbbb20640 1 -- 172.21.15.37:0/2614180972 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "recovery_fs_metadata", "pool2": "recovery_fs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7febb40b34c0 con 0x7febb40b3a10 2024-09-17T04:34:43.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.032+0000 7febaa7fc640 1 -- 172.21.15.37:0/2614180972 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "recovery_fs_metadata", "pool2": "recovery_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'recovery_fs_metadata' removed v44) ==== 196+0+0 (secure 0 0 0) 0x7febb00624b0 con 0x7febb40b3a10 2024-09-17T04:34:43.032 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'recovery_fs_metadata' removed 2024-09-17T04:34:43.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.036+0000 7febbbb20640 1 -- 172.21.15.37:0/2614180972 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7feb98078bc0 msgr2=0x7feb9807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:43.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.036+0000 7febbbb20640 1 --2- 172.21.15.37:0/2614180972 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7feb98078bc0 0x7feb9807b0d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7feba4034000 tx=0x7feba4004730 comp rx=0 tx=0).stop 2024-09-17T04:34:43.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.036+0000 7febbbb20640 1 -- 172.21.15.37:0/2614180972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb40b3a10 msgr2=0x7febb40c9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:43.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.036+0000 7febbbb20640 1 --2- 172.21.15.37:0/2614180972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb40b3a10 0x7febb40c9090 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7febb000c3c0 tx=0x7febb000c880 comp rx=0 tx=0).stop 2024-09-17T04:34:43.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.038+0000 7febbbb20640 1 -- 172.21.15.37:0/2614180972 shutdown_connections 2024-09-17T04:34:43.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.038+0000 7febbbb20640 1 --2- 172.21.15.37:0/2614180972 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7feb98078bc0 0x7feb9807b0d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:43.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.038+0000 7febbbb20640 1 --2- 172.21.15.37:0/2614180972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febb40c7ad0 0x7febb41f8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:43.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.038+0000 7febbbb20640 1 --2- 172.21.15.37:0/2614180972 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7febb40c9d70 0x7febb40cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:43.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.038+0000 7febbbb20640 1 --2- 172.21.15.37:0/2614180972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb40b3a10 0x7febb40c9090 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:43.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.038+0000 7febbbb20640 1 -- 172.21.15.37:0/2614180972 >> 172.21.15.37:0/2614180972 conn(0x7febb408bef0 msgr2=0x7febb40d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:43.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.039+0000 7febbbb20640 1 -- 172.21.15.37:0/2614180972 shutdown_connections 2024-09-17T04:34:43.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.040+0000 7febbbb20640 1 -- 172.21.15.37:0/2614180972 wait complete. 2024-09-17T04:34:43.054 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T04:34:43.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.182+0000 7fc90e863640 1 -- 172.21.15.37:0/4152645996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9080b6750 msgr2=0x7fc908161ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:43.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.182+0000 7fc90e863640 1 --2- 172.21.15.37:0/4152645996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9080b6750 0x7fc908161ff0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc0098e0 tx=0x7fc8fc02a530 comp rx=0 tx=0).stop 2024-09-17T04:34:43.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.183+0000 7fc90e863640 1 -- 172.21.15.37:0/4152645996 shutdown_connections 2024-09-17T04:34:43.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.183+0000 7fc90e863640 1 --2- 172.21.15.37:0/4152645996 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc9080b3a10 0x7fc9080d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:43.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.183+0000 7fc90e863640 1 --2- 172.21.15.37:0/4152645996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9080b6750 0x7fc908161ff0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:43.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.183+0000 7fc90e863640 1 -- 172.21.15.37:0/4152645996 >> 172.21.15.37:0/4152645996 conn(0x7fc90808bef0 msgr2=0x7fc9080d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:43.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.183+0000 7fc90e863640 1 -- 172.21.15.37:0/4152645996 shutdown_connections 2024-09-17T04:34:43.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.183+0000 7fc90e863640 1 -- 172.21.15.37:0/4152645996 wait complete. 2024-09-17T04:34:43.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.184+0000 7fc90e863640 1 Processor -- start 2024-09-17T04:34:43.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.185+0000 7fc90e863640 1 -- start start 2024-09-17T04:34:43.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.186+0000 7fc90e863640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9080b3a10 0x7fc9080c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:43.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.186+0000 7fc90e863640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc9080cd530 0x7fc9080cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:43.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.186+0000 7fc90e863640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9080cde80 0x7fc9081f8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:43.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.186+0000 7fc90e863640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc9080d2e30 con 0x7fc9080b3a10 2024-09-17T04:34:43.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.186+0000 7fc90e863640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc9080d2cb0 con 0x7fc9080cde80 2024-09-17T04:34:43.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.186+0000 7fc90e863640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fc9080d2fb0 con 0x7fc9080cd530 2024-09-17T04:34:43.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.186+0000 7fc907fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9080b3a10 0x7fc9080c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:43.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.186+0000 7fc907fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9080b3a10 0x7fc9080c8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33274/0 (socket says 172.21.15.37:33274) 2024-09-17T04:34:43.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.186+0000 7fc907fff640 1 -- 172.21.15.37:0/983195587 learned_addr learned my addr 172.21.15.37:0/983195587 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:43.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.186+0000 7fc9077fe640 1 --2- 172.21.15.37:0/983195587 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc9080cd530 0x7fc9080cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:43.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.186+0000 7fc90ce00640 1 --2- 172.21.15.37:0/983195587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9080cde80 0x7fc9081f8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:43.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.186+0000 7fc907fff640 1 -- 172.21.15.37:0/983195587 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc9080cd530 msgr2=0x7fc9080cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:43.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.186+0000 7fc907fff640 1 --2- 172.21.15.37:0/983195587 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc9080cd530 0x7fc9080cd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:43.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.186+0000 7fc907fff640 1 -- 172.21.15.37:0/983195587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9080cde80 msgr2=0x7fc9081f8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:43.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.186+0000 7fc907fff640 1 --2- 172.21.15.37:0/983195587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9080cde80 0x7fc9081f8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:43.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.186+0000 7fc907fff640 1 -- 172.21.15.37:0/983195587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9080c89c0 con 0x7fc9080b3a10 2024-09-17T04:34:43.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.187+0000 7fc907fff640 1 --2- 172.21.15.37:0/983195587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9080b3a10 0x7fc9080c8480 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800b3f0 tx=0x7fc8f800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:43.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.187+0000 7fc904ff9640 1 -- 172.21.15.37:0/983195587 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8015020 con 0x7fc9080b3a10 2024-09-17T04:34:43.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.187+0000 7fc904ff9640 1 -- 172.21.15.37:0/983195587 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8f8011920 con 0x7fc9080b3a10 2024-09-17T04:34:43.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.187+0000 7fc904ff9640 1 -- 172.21.15.37:0/983195587 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8011d70 con 0x7fc9080b3a10 2024-09-17T04:34:43.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.187+0000 7fc90e863640 1 -- 172.21.15.37:0/983195587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9080c8e90 con 0x7fc9080b3a10 2024-09-17T04:34:43.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.187+0000 7fc90e863640 1 -- 172.21.15.37:0/983195587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9080c9450 con 0x7fc9080b3a10 2024-09-17T04:34:43.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.188+0000 7fc90e863640 1 -- 172.21.15.37:0/983195587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9080d34f0 con 0x7fc9080b3a10 2024-09-17T04:34:43.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.189+0000 7fc904ff9640 1 -- 172.21.15.37:0/983195587 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc8f8018020 con 0x7fc9080b3a10 2024-09-17T04:34:43.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.189+0000 7fc904ff9640 1 --2- 172.21.15.37:0/983195587 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc8f4078b90 0x7fc8f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:43.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.189+0000 7fc904ff9640 1 -- 172.21.15.37:0/983195587 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5448+0+0 (secure 0 0 0) 0x7fc8f809a6d0 con 0x7fc9080b3a10 2024-09-17T04:34:43.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.189+0000 7fc9077fe640 1 --2- 172.21.15.37:0/983195587 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc8f4078b90 0x7fc8f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:43.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.190+0000 7fc9077fe640 1 --2- 172.21.15.37:0/983195587 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc8f4078b90 0x7fc8f407b0a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc034000 tx=0x7fc8fc004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:43.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.194+0000 7fc904ff9640 1 -- 172.21.15.37:0/983195587 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc8f801f030 con 0x7fc9080b3a10 2024-09-17T04:34:43.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:43.379+0000 7fc90e863640 1 -- 172.21.15.37:0/983195587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc908076310 con 0x7fc9080b3a10 2024-09-17T04:34:44.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.033+0000 7fc904ff9640 1 -- 172.21.15.37:0/983195587 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v45) ==== 169+0+0 (secure 0 0 0) 0x7fc8f8062650 con 0x7fc9080b3a10 2024-09-17T04:34:44.033 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-09-17T04:34:44.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.037+0000 7fc90e863640 1 -- 172.21.15.37:0/983195587 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc8f4078b90 msgr2=0x7fc8f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:44.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.037+0000 7fc90e863640 1 --2- 172.21.15.37:0/983195587 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc8f4078b90 0x7fc8f407b0a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc034000 tx=0x7fc8fc004730 comp rx=0 tx=0).stop 2024-09-17T04:34:44.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.037+0000 7fc90e863640 1 -- 172.21.15.37:0/983195587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9080b3a10 msgr2=0x7fc9080c8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:44.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.037+0000 7fc90e863640 1 --2- 172.21.15.37:0/983195587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9080b3a10 0x7fc9080c8480 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800b3f0 tx=0x7fc8f800b8b0 comp rx=0 tx=0).stop 2024-09-17T04:34:44.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.038+0000 7fc90e863640 1 -- 172.21.15.37:0/983195587 shutdown_connections 2024-09-17T04:34:44.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.038+0000 7fc90e863640 1 --2- 172.21.15.37:0/983195587 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc8f4078b90 0x7fc8f407b0a0 secure :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc034000 tx=0x7fc8fc004730 comp rx=0 tx=0).stop 2024-09-17T04:34:44.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.038+0000 7fc90e863640 1 --2- 172.21.15.37:0/983195587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9080cde80 0x7fc9081f8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:44.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.038+0000 7fc90e863640 1 --2- 172.21.15.37:0/983195587 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc9080cd530 0x7fc9080cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:44.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.038+0000 7fc90e863640 1 --2- 172.21.15.37:0/983195587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9080b3a10 0x7fc9080c8480 secure :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800b3f0 tx=0x7fc8f800b8b0 comp rx=0 tx=0).stop 2024-09-17T04:34:44.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.038+0000 7fc90e863640 1 -- 172.21.15.37:0/983195587 >> 172.21.15.37:0/983195587 conn(0x7fc90808bef0 msgr2=0x7fc90815ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:44.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.038+0000 7fc90e863640 1 -- 172.21.15.37:0/983195587 shutdown_connections 2024-09-17T04:34:44.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.038+0000 7fc90e863640 1 -- 172.21.15.37:0/983195587 wait complete. 2024-09-17T04:34:44.057 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T04:34:44.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.185+0000 7fc82b43f640 1 -- 172.21.15.37:0/4163401966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8240b77a0 msgr2=0x7fc8240cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:44.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.185+0000 7fc82b43f640 1 --2- 172.21.15.37:0/4163401966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8240b77a0 0x7fc8240cddc0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc824069cb0 tx=0x7fc81c02a560 comp rx=0 tx=0).stop 2024-09-17T04:34:44.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.185+0000 7fc82b43f640 1 -- 172.21.15.37:0/4163401966 shutdown_connections 2024-09-17T04:34:44.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.185+0000 7fc82b43f640 1 --2- 172.21.15.37:0/4163401966 >> v2:172.21.15.92:3300/0 conn(0x7fc8240cea60 0x7fc8240cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:44.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.185+0000 7fc82b43f640 1 --2- 172.21.15.37:0/4163401966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8240b77a0 0x7fc8240cddc0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:44.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.185+0000 7fc82b43f640 1 -- 172.21.15.37:0/4163401966 >> 172.21.15.37:0/4163401966 conn(0x7fc8240d2470 msgr2=0x7fc8240d2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:44.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.185+0000 7fc82b43f640 1 -- 172.21.15.37:0/4163401966 shutdown_connections 2024-09-17T04:34:44.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.185+0000 7fc82b43f640 1 -- 172.21.15.37:0/4163401966 wait complete. 2024-09-17T04:34:44.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.187+0000 7fc82b43f640 1 Processor -- start 2024-09-17T04:34:44.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.187+0000 7fc82b43f640 1 -- start start 2024-09-17T04:34:44.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.189+0000 7fc82b43f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8240b1e00 0x7fc82415a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:44.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.189+0000 7fc82b43f640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc8240b77a0 0x7fc824164d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:44.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.189+0000 7fc82b43f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8240cea60 0x7fc8241652b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:44.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.189+0000 7fc82b43f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc8240ce3b0 con 0x7fc8240b1e00 2024-09-17T04:34:44.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.189+0000 7fc82b43f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc8240b2f70 con 0x7fc8240cea60 2024-09-17T04:34:44.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.189+0000 7fc82b43f640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fc8240b30f0 con 0x7fc8240b77a0 2024-09-17T04:34:44.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.189+0000 7fc8291db640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8240b1e00 0x7fc82415a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:44.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.189+0000 7fc8291db640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8240b1e00 0x7fc82415a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33288/0 (socket says 172.21.15.37:33288) 2024-09-17T04:34:44.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.189+0000 7fc8291db640 1 -- 172.21.15.37:0/1364095000 learned_addr learned my addr 172.21.15.37:0/1364095000 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:44.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.189+0000 7fc8299dc640 1 --2- 172.21.15.37:0/1364095000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8240cea60 0x7fc8241652b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:44.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.189+0000 7fc8289da640 1 --2- 172.21.15.37:0/1364095000 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc8240b77a0 0x7fc824164d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:44.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.189+0000 7fc8291db640 1 -- 172.21.15.37:0/1364095000 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc8240b77a0 msgr2=0x7fc824164d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:44.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.189+0000 7fc8291db640 1 --2- 172.21.15.37:0/1364095000 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc8240b77a0 0x7fc824164d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:44.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.189+0000 7fc8291db640 1 -- 172.21.15.37:0/1364095000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8240cea60 msgr2=0x7fc8241652b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:44.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.189+0000 7fc8291db640 1 --2- 172.21.15.37:0/1364095000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8240cea60 0x7fc8241652b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:44.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.189+0000 7fc8291db640 1 -- 172.21.15.37:0/1364095000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82415f050 con 0x7fc8240b1e00 2024-09-17T04:34:44.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.189+0000 7fc8291db640 1 --2- 172.21.15.37:0/1364095000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8240b1e00 0x7fc82415a390 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc81c02ab50 tx=0x7fc81c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:44.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.190+0000 7fc821ffb640 1 -- 172.21.15.37:0/1364095000 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c031d60 con 0x7fc8240b1e00 2024-09-17T04:34:44.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.190+0000 7fc821ffb640 1 -- 172.21.15.37:0/1364095000 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc81c033e10 con 0x7fc8240b1e00 2024-09-17T04:34:44.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.190+0000 7fc82b43f640 1 -- 172.21.15.37:0/1364095000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc82415f330 con 0x7fc8240b1e00 2024-09-17T04:34:44.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.190+0000 7fc821ffb640 1 -- 172.21.15.37:0/1364095000 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c033330 con 0x7fc8240b1e00 2024-09-17T04:34:44.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.190+0000 7fc82b43f640 1 -- 172.21.15.37:0/1364095000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc82415f8f0 con 0x7fc8240b1e00 2024-09-17T04:34:44.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.191+0000 7fc82b43f640 1 -- 172.21.15.37:0/1364095000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8240cddc0 con 0x7fc8240b1e00 2024-09-17T04:34:44.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.192+0000 7fc821ffb640 1 -- 172.21.15.37:0/1364095000 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc81c0334d0 con 0x7fc8240b1e00 2024-09-17T04:34:44.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.192+0000 7fc821ffb640 1 --2- 172.21.15.37:0/1364095000 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc810078b90 0x7fc81007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:44.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.192+0000 7fc821ffb640 1 -- 172.21.15.37:0/1364095000 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5029+0+0 (secure 0 0 0) 0x7fc81c0bbd60 con 0x7fc8240b1e00 2024-09-17T04:34:44.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.192+0000 7fc8289da640 1 --2- 172.21.15.37:0/1364095000 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc810078b90 0x7fc81007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:44.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.193+0000 7fc8289da640 1 --2- 172.21.15.37:0/1364095000 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc810078b90 0x7fc81007b0a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc818000f60 tx=0x7fc818009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:44.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.197+0000 7fc821ffb640 1 -- 172.21.15.37:0/1364095000 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc81c03f030 con 0x7fc8240b1e00 2024-09-17T04:34:44.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.381+0000 7fc82b43f640 1 -- 172.21.15.37:0/1364095000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fc824070190 con 0x7fc8240b1e00 2024-09-17T04:34:44.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.384+0000 7fc821ffb640 1 -- 172.21.15.37:0/1364095000 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v33) ==== 71+0+0 (secure 0 0 0) 0x7fc81c083e00 con 0x7fc8240b1e00 2024-09-17T04:34:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.387+0000 7fc82b43f640 1 -- 172.21.15.37:0/1364095000 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc810078b90 msgr2=0x7fc81007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.387+0000 7fc82b43f640 1 --2- 172.21.15.37:0/1364095000 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc810078b90 0x7fc81007b0a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc818000f60 tx=0x7fc818009040 comp rx=0 tx=0).stop 2024-09-17T04:34:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.387+0000 7fc82b43f640 1 -- 172.21.15.37:0/1364095000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8240b1e00 msgr2=0x7fc82415a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.387+0000 7fc82b43f640 1 --2- 172.21.15.37:0/1364095000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8240b1e00 0x7fc82415a390 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc81c02ab50 tx=0x7fc81c033c20 comp rx=0 tx=0).stop 2024-09-17T04:34:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.387+0000 7fc82b43f640 1 -- 172.21.15.37:0/1364095000 shutdown_connections 2024-09-17T04:34:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.387+0000 7fc82b43f640 1 --2- 172.21.15.37:0/1364095000 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc810078b90 0x7fc81007b0a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.387+0000 7fc82b43f640 1 --2- 172.21.15.37:0/1364095000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8240cea60 0x7fc8241652b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.387+0000 7fc82b43f640 1 --2- 172.21.15.37:0/1364095000 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc8240b77a0 0x7fc824164d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.387+0000 7fc82b43f640 1 --2- 172.21.15.37:0/1364095000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8240b1e00 0x7fc82415a390 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.387+0000 7fc82b43f640 1 -- 172.21.15.37:0/1364095000 >> 172.21.15.37:0/1364095000 conn(0x7fc8240d2470 msgr2=0x7fc8240d2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.387+0000 7fc82b43f640 1 -- 172.21.15.37:0/1364095000 shutdown_connections 2024-09-17T04:34:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.387+0000 7fc82b43f640 1 -- 172.21.15.37:0/1364095000 wait complete. 2024-09-17T04:34:44.397 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_recovery_pool.TestRecoveryPool.test_rebuild_simple' 2024-09-17T04:34:44.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.515+0000 7f65fee71640 1 -- 172.21.15.37:0/176538071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65f8072a90 msgr2=0x7f65f80cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:44.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.515+0000 7f65fee71640 1 --2- 172.21.15.37:0/176538071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65f8072a90 0x7f65f80cf050 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009820 tx=0x7f65e802a5a0 comp rx=0 tx=0).stop 2024-09-17T04:34:44.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.515+0000 7f65fee71640 1 -- 172.21.15.37:0/176538071 shutdown_connections 2024-09-17T04:34:44.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.515+0000 7f65fee71640 1 --2- 172.21.15.37:0/176538071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65f8072a90 0x7f65f80cf050 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:44.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.515+0000 7f65fee71640 1 -- 172.21.15.37:0/176538071 >> 172.21.15.37:0/176538071 conn(0x7f65f80d2460 msgr2=0x7f65f80d2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:44.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.515+0000 7f65fee71640 1 -- 172.21.15.37:0/176538071 shutdown_connections 2024-09-17T04:34:44.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.515+0000 7f65fee71640 1 -- 172.21.15.37:0/176538071 wait complete. 2024-09-17T04:34:44.516 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:34:44.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.516+0000 7f65fee71640 1 Processor -- start 2024-09-17T04:34:44.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.517+0000 7f65fee71640 1 -- start start 2024-09-17T04:34:44.518 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:34:44.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.518+0000 7f65fee71640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f65f8072a90 0x7f65f8154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:44.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.518+0000 7f65fee71640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65f80cd1b0 0x7f65f80b0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:44.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.518+0000 7f65fee71640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65f80cf590 0x7f65f80b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:44.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.518+0000 7f65fee71640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f65f80b7d80 con 0x7f65f80cf590 2024-09-17T04:34:44.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.518+0000 7f65fee71640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f65f80b7c00 con 0x7f65f80cd1b0 2024-09-17T04:34:44.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.518+0000 7f65fee71640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f65f8168350 con 0x7f65f8072a90 2024-09-17T04:34:44.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.518+0000 7f65fd40e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65f80cf590 0x7f65f80b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:44.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.518+0000 7f65fd40e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65f80cf590 0x7f65f80b05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33298/0 (socket says 172.21.15.37:33298) 2024-09-17T04:34:44.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.518+0000 7f65fd40e640 1 -- 172.21.15.37:0/1039409404 learned_addr learned my addr 172.21.15.37:0/1039409404 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:44.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.518+0000 7f65f7fff640 1 --2- 172.21.15.37:0/1039409404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65f80cd1b0 0x7f65f80b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:44.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.518+0000 7f65fcc0d640 1 --2- 172.21.15.37:0/1039409404 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f65f8072a90 0x7f65f8154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:44.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.518+0000 7f65fd40e640 1 -- 172.21.15.37:0/1039409404 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f65f8072a90 msgr2=0x7f65f8154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:44.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.518+0000 7f65fd40e640 1 --2- 172.21.15.37:0/1039409404 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f65f8072a90 0x7f65f8154bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:44.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.518+0000 7f65fd40e640 1 -- 172.21.15.37:0/1039409404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65f80cd1b0 msgr2=0x7f65f80b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:44.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.518+0000 7f65fd40e640 1 --2- 172.21.15.37:0/1039409404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65f80cd1b0 0x7f65f80b0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:44.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.518+0000 7f65fd40e640 1 -- 172.21.15.37:0/1039409404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f80b3960 con 0x7f65f80cf590 2024-09-17T04:34:44.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.518+0000 7f65f7fff640 1 --2- 172.21.15.37:0/1039409404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65f80cd1b0 0x7f65f80b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:44.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.519+0000 7f65fd40e640 1 --2- 172.21.15.37:0/1039409404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65f80cf590 0x7f65f80b05c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f65f000c730 tx=0x7f65f000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:44.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.519+0000 7f65f57fa640 1 -- 172.21.15.37:0/1039409404 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f0003b70 con 0x7f65f80cf590 2024-09-17T04:34:44.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.519+0000 7f65f57fa640 1 -- 172.21.15.37:0/1039409404 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65f0003d10 con 0x7f65f80cf590 2024-09-17T04:34:44.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.519+0000 7f65fee71640 1 -- 172.21.15.37:0/1039409404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f80b3c40 con 0x7f65f80cf590 2024-09-17T04:34:44.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.519+0000 7f65f57fa640 1 -- 172.21.15.37:0/1039409404 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f0005330 con 0x7f65f80cf590 2024-09-17T04:34:44.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.519+0000 7f65fee71640 1 -- 172.21.15.37:0/1039409404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f80b4200 con 0x7f65f80cf590 2024-09-17T04:34:44.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.520+0000 7f65fee71640 1 -- 172.21.15.37:0/1039409404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65f80cf050 con 0x7f65f80cf590 2024-09-17T04:34:44.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.521+0000 7f65f57fa640 1 -- 172.21.15.37:0/1039409404 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f65f00054d0 con 0x7f65f80cf590 2024-09-17T04:34:44.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.521+0000 7f65f57fa640 1 --2- 172.21.15.37:0/1039409404 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f65e4078bc0 0x7f65e407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:44.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.521+0000 7f65f57fa640 1 -- 172.21.15.37:0/1039409404 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5029+0+0 (secure 0 0 0) 0x7f65f009a780 con 0x7f65f80cf590 2024-09-17T04:34:44.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.522+0000 7f65fcc0d640 1 --2- 172.21.15.37:0/1039409404 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f65e4078bc0 0x7f65e407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:44.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.522+0000 7f65fcc0d640 1 --2- 172.21.15.37:0/1039409404 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f65e4078bc0 0x7f65e407b0d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f65e8034000 tx=0x7f65e8004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:44.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.527+0000 7f65f57fa640 1 -- 172.21.15.37:0/1039409404 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f65f0062e70 con 0x7f65f80cf590 2024-09-17T04:34:44.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:44.785+0000 7f65fee71640 1 -- 172.21.15.37:0/1039409404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_recovery_pool.TestRecoveryPool.test_rebuild_simple"]} v 0) -- 0x7f65f806a150 con 0x7f65f80cf590 2024-09-17T04:34:45.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.043+0000 7f65f57fa640 1 -- 172.21.15.37:0/1039409404 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_recovery_pool.TestRecoveryPool.test_rebuild_simple"]}]=0 v104) ==== 147+0+0 (secure 0 0 0) 0x7f65f0062770 con 0x7f65f80cf590 2024-09-17T04:34:45.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.046+0000 7f65fee71640 1 -- 172.21.15.37:0/1039409404 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f65e4078bc0 msgr2=0x7f65e407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:45.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.046+0000 7f65fee71640 1 --2- 172.21.15.37:0/1039409404 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f65e4078bc0 0x7f65e407b0d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f65e8034000 tx=0x7f65e8004730 comp rx=0 tx=0).stop 2024-09-17T04:34:45.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.046+0000 7f65fee71640 1 -- 172.21.15.37:0/1039409404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65f80cf590 msgr2=0x7f65f80b05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:45.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.046+0000 7f65fee71640 1 --2- 172.21.15.37:0/1039409404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65f80cf590 0x7f65f80b05c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f65f000c730 tx=0x7f65f000cb30 comp rx=0 tx=0).stop 2024-09-17T04:34:45.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.046+0000 7f65fee71640 1 -- 172.21.15.37:0/1039409404 shutdown_connections 2024-09-17T04:34:45.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.046+0000 7f65fee71640 1 --2- 172.21.15.37:0/1039409404 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f65e4078bc0 0x7f65e407b0d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:45.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.046+0000 7f65fee71640 1 --2- 172.21.15.37:0/1039409404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65f80cf590 0x7f65f80b05c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:45.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.046+0000 7f65fee71640 1 --2- 172.21.15.37:0/1039409404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65f80cd1b0 0x7f65f80b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:45.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.046+0000 7f65fee71640 1 --2- 172.21.15.37:0/1039409404 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f65f8072a90 0x7f65f8154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:45.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.046+0000 7f65fee71640 1 -- 172.21.15.37:0/1039409404 >> 172.21.15.37:0/1039409404 conn(0x7f65f80d2460 msgr2=0x7f65f8160690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:45.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.046+0000 7f65fee71640 1 -- 172.21.15.37:0/1039409404 shutdown_connections 2024-09-17T04:34:45.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.046+0000 7f65fee71640 1 -- 172.21.15.37:0/1039409404 wait complete. 2024-09-17T04:34:45.058 INFO:tasks.cephfs_test_runner:test_rebuild_simple (tasks.cephfs.test_recovery_pool.TestRecoveryPool) ... ok 2024-09-17T04:34:45.058 INFO:tasks.cephfs_test_runner: 2024-09-17T04:34:45.058 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-17T04:34:45.058 INFO:tasks.cephfs_test_runner:Ran 1 test in 64.682s 2024-09-17T04:34:45.058 INFO:tasks.cephfs_test_runner: 2024-09-17T04:34:45.058 INFO:tasks.cephfs_test_runner:OK 2024-09-17T04:34:45.059 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-17T04:34:45.070 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-17T04:34:45.116 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-17T04:34:45.116 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:34:45.116 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:34:45.133 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17600 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:34:45.133 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T04:34:45.134 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:34:45.190 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17600 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T04:34:45.190 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:34:45.190 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:34:45.209 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056024,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497460k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=824 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497460k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:34:45.210 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-17T04:34:45.210 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T04:34:45.268 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056024,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497460k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=824 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497460k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T04:34:45.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:45.269 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-09-17T04:34:45.285 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T04:34:45.285 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T04:34:45.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:45.286 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T04:34:45.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:45.353 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T04:34:45.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:45.422 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-09-17T04:34:45.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:45.518 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2024-09-17T04:34:45.537 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T04:34:45.537 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T04:34:45.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:45.537 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T04:34:45.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:45.607 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T04:34:45.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T04:34:45.679 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2024-09-17T04:34:45.785 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-17T04:34:45.794 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T04:34:45.794 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:34:45.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.914+0000 7f140ca30640 1 -- 172.21.15.37:0/391667210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14080ca890 msgr2=0x7f14080caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:45.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.914+0000 7f140ca30640 1 --2- 172.21.15.37:0/391667210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14080ca890 0x7f14080caca0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009830 tx=0x7f13fc02a510 comp rx=0 tx=0).stop 2024-09-17T04:34:45.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.915+0000 7f140ca30640 1 -- 172.21.15.37:0/391667210 shutdown_connections 2024-09-17T04:34:45.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.915+0000 7f140ca30640 1 --2- 172.21.15.37:0/391667210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14080ca890 0x7f14080caca0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:45.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.915+0000 7f140ca30640 1 --2- 172.21.15.37:0/391667210 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f14080af7a0 0x7f14080b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:45.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.915+0000 7f140ca30640 1 -- 172.21.15.37:0/391667210 >> 172.21.15.37:0/391667210 conn(0x7f140806d3f0 msgr2=0x7f14080d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:45.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.915+0000 7f140ca30640 1 -- 172.21.15.37:0/391667210 shutdown_connections 2024-09-17T04:34:45.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.915+0000 7f140ca30640 1 -- 172.21.15.37:0/391667210 wait complete. 2024-09-17T04:34:45.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.917+0000 7f140ca30640 1 Processor -- start 2024-09-17T04:34:45.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.917+0000 7f140ca30640 1 -- start start 2024-09-17T04:34:45.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.918+0000 7f140ca30640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14080af7a0 0x7f14081e5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:45.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.918+0000 7f140ca30640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14080ca890 0x7f1408159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:45.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.918+0000 7f140ca30640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f14080cb1e0 0x7f14081597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:45.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.918+0000 7f140ca30640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f140806fae0 con 0x7f14080ca890 2024-09-17T04:34:45.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.918+0000 7f140ca30640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f140806f960 con 0x7f14080af7a0 2024-09-17T04:34:45.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.918+0000 7f140ca30640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f140806fc60 con 0x7f14080cb1e0 2024-09-17T04:34:45.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.918+0000 7f1405d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14080ca890 0x7f1408159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:45.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.918+0000 7f1405d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14080ca890 0x7f1408159270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39586/0 (socket says 172.21.15.37:39586) 2024-09-17T04:34:45.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.918+0000 7f1405d9b640 1 -- 172.21.15.37:0/1775550720 learned_addr learned my addr 172.21.15.37:0/1775550720 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:45.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.918+0000 7f140659c640 1 --2- 172.21.15.37:0/1775550720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14080af7a0 0x7f14081e5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:45.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.918+0000 7f1406d9d640 1 --2- 172.21.15.37:0/1775550720 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f14080cb1e0 0x7f14081597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:45.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.918+0000 7f1405d9b640 1 -- 172.21.15.37:0/1775550720 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f14080cb1e0 msgr2=0x7f14081597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:45.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.918+0000 7f1405d9b640 1 --2- 172.21.15.37:0/1775550720 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f14080cb1e0 0x7f14081597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:45.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.918+0000 7f1405d9b640 1 -- 172.21.15.37:0/1775550720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14080af7a0 msgr2=0x7f14081e5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:45.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.918+0000 7f1405d9b640 1 --2- 172.21.15.37:0/1775550720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14080af7a0 0x7f14081e5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:45.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.918+0000 7f1405d9b640 1 -- 172.21.15.37:0/1775550720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14081e4010 con 0x7f14080ca890 2024-09-17T04:34:45.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.918+0000 7f140659c640 1 --2- 172.21.15.37:0/1775550720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14080af7a0 0x7f14081e5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:45.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.919+0000 7f1405d9b640 1 --2- 172.21.15.37:0/1775550720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14080ca890 0x7f1408159270 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009950 tx=0x7f13fc031610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:45.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.919+0000 7f13f6ffd640 1 -- 172.21.15.37:0/1775550720 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc003a80 con 0x7f14080ca890 2024-09-17T04:34:45.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.919+0000 7f13f6ffd640 1 -- 172.21.15.37:0/1775550720 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13fc003d10 con 0x7f14080ca890 2024-09-17T04:34:45.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.919+0000 7f13f6ffd640 1 -- 172.21.15.37:0/1775550720 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc03d690 con 0x7f14080ca890 2024-09-17T04:34:45.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.919+0000 7f140ca30640 1 -- 172.21.15.37:0/1775550720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14081e6030 con 0x7f14080ca890 2024-09-17T04:34:45.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.919+0000 7f140ca30640 1 -- 172.21.15.37:0/1775550720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14081e45b0 con 0x7f14080ca890 2024-09-17T04:34:45.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.920+0000 7f140ca30640 1 -- 172.21.15.37:0/1775550720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14081e5da0 con 0x7f14080ca890 2024-09-17T04:34:45.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.921+0000 7f13f6ffd640 1 -- 172.21.15.37:0/1775550720 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f13fc03d830 con 0x7f14080ca890 2024-09-17T04:34:45.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.921+0000 7f13f6ffd640 1 --2- 172.21.15.37:0/1775550720 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f13e8078b90 0x7f13e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:45.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.921+0000 7f13f6ffd640 1 -- 172.21.15.37:0/1775550720 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5029+0+0 (secure 0 0 0) 0x7f13fc0be540 con 0x7f14080ca890 2024-09-17T04:34:45.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.922+0000 7f140659c640 1 --2- 172.21.15.37:0/1775550720 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f13e8078b90 0x7f13e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:45.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.922+0000 7f140659c640 1 --2- 172.21.15.37:0/1775550720 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f13e8078b90 0x7f13e807b0a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f13f8005fd0 tx=0x7f13f800f360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:45.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:45.926+0000 7f13f6ffd640 1 -- 172.21.15.37:0/1775550720 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f13fc04c030 con 0x7f14080ca890 2024-09-17T04:34:46.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.107+0000 7f140ca30640 1 -- 172.21.15.37:0/1775550720 --> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1408070110 con 0x7f13e8078b90 2024-09-17T04:34:46.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.108+0000 7f13f6ffd640 1 -- 172.21.15.37:0/1775550720 <== mgr.4100 v2:172.21.15.92:6800/864337835 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13481 (secure 0 0 0) 0x7f1408070110 con 0x7f13e8078b90 2024-09-17T04:34:46.109 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:46.109 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-17T04:34:46.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.110+0000 7f140ca30640 1 -- 172.21.15.37:0/1775550720 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f13e8078b90 msgr2=0x7f13e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:46.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.110+0000 7f140ca30640 1 --2- 172.21.15.37:0/1775550720 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f13e8078b90 0x7f13e807b0a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f13f8005fd0 tx=0x7f13f800f360 comp rx=0 tx=0).stop 2024-09-17T04:34:46.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.110+0000 7f140ca30640 1 -- 172.21.15.37:0/1775550720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14080ca890 msgr2=0x7f1408159270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:46.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.110+0000 7f140ca30640 1 --2- 172.21.15.37:0/1775550720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14080ca890 0x7f1408159270 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009950 tx=0x7f13fc031610 comp rx=0 tx=0).stop 2024-09-17T04:34:46.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.111+0000 7f140ca30640 1 -- 172.21.15.37:0/1775550720 shutdown_connections 2024-09-17T04:34:46.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.111+0000 7f140ca30640 1 --2- 172.21.15.37:0/1775550720 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f13e8078b90 0x7f13e807b0a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:46.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.111+0000 7f140ca30640 1 --2- 172.21.15.37:0/1775550720 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f14080cb1e0 0x7f14081597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:46.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.111+0000 7f140ca30640 1 --2- 172.21.15.37:0/1775550720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14080ca890 0x7f1408159270 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:46.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.111+0000 7f140ca30640 1 --2- 172.21.15.37:0/1775550720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14080af7a0 0x7f14081e5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:46.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.111+0000 7f140ca30640 1 -- 172.21.15.37:0/1775550720 >> 172.21.15.37:0/1775550720 conn(0x7f140806d3f0 msgr2=0x7f140815b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:46.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.111+0000 7f140ca30640 1 -- 172.21.15.37:0/1775550720 shutdown_connections 2024-09-17T04:34:46.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.111+0000 7f140ca30640 1 -- 172.21.15.37:0/1775550720 wait complete. 2024-09-17T04:34:46.122 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-09-17T04:34:44.466429+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":218,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237844,"kb_used_data":17440,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754736876,"statfs":{"total":773094113280,"available":772850561024,"internally_reserved":0,"allocated":17858560,"data_stored":15471022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6322167,"num_objects":-15,"num_object_clones":0,"num_object_copies":-30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-15,"num_whiteouts":0,"num_read":-53,"num_read_kb":-39,"num_write":-46,"num_write_kb":-6192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.913078"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":151,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T04:34:41.150326+0000","last_change":"2024-09-17T04:33:09.978742+0000","last_active":"2024-09-17T04:34:41.150326+0000","last_peered":"2024-09-17T04:34:41.150326+0000","last_clean":"2024-09-17T04:34:41.150326+0000","last_became_active":"2024-09-17T04:33:09.977496+0000","last_became_peered":"2024-09-17T04:33:09.977496+0000","last_unstale":"2024-09-17T04:34:41.150326+0000","last_undegraded":"2024-09-17T04:34:41.150326+0000","last_fullsized":"2024-09-17T04:34:41.150326+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:08.930340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:08.930340+0000","last_clean_scrub_stamp":"2024-09-17T04:33:08.930340+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:21:26.566102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542167,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30144,"kb_used_data":3116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341696,"statfs":{"total":96636764160,"available":96605896704,"internally_reserved":0,"allocated":3190784,"data_stored":2893354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542167,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29568,"kb_used_data":2540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342272,"statfs":{"total":96636764160,"available":96606486528,"internally_reserved":0,"allocated":2600960,"data_stored":2303568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542168,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":4584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":4694016,"data_stored":4400118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542167,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":536576,"data_stored":235657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542168,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":507904,"data_stored":206382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542167,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":540672,"data_stored":235679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542166,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":4584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":4694016,"data_stored":4400118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542167,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28100,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343740,"statfs":{"total":96636764160,"available":96607989760,"internally_reserved":0,"allocated":1093632,"data_stored":796146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:34:46.122 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:34:46.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.239+0000 7f4e39a4a640 1 -- 172.21.15.37:0/543226152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e3406f9f0 msgr2=0x7f4e340d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:46.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.239+0000 7f4e39a4a640 1 --2- 172.21.15.37:0/543226152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e3406f9f0 0x7f4e340d0f00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4e340b33c0 tx=0x7f4e2402a5f0 comp rx=0 tx=0).stop 2024-09-17T04:34:46.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.239+0000 7f4e39a4a640 1 -- 172.21.15.37:0/543226152 shutdown_connections 2024-09-17T04:34:46.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.239+0000 7f4e39a4a640 1 --2- 172.21.15.37:0/543226152 >> v2:172.21.15.92:3300/0 conn(0x7f4e340d1440 0x7f4e34162d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:46.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.239+0000 7f4e39a4a640 1 --2- 172.21.15.37:0/543226152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e3406f9f0 0x7f4e340d0f00 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:46.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.239+0000 7f4e39a4a640 1 -- 172.21.15.37:0/543226152 >> 172.21.15.37:0/543226152 conn(0x7f4e3408bef0 msgr2=0x7f4e340d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:46.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.239+0000 7f4e39a4a640 1 -- 172.21.15.37:0/543226152 shutdown_connections 2024-09-17T04:34:46.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.239+0000 7f4e39a4a640 1 -- 172.21.15.37:0/543226152 wait complete. 2024-09-17T04:34:46.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.241+0000 7f4e39a4a640 1 Processor -- start 2024-09-17T04:34:46.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.241+0000 7f4e39a4a640 1 -- start start 2024-09-17T04:34:46.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.242+0000 7f4e39a4a640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4e3406f9f0 0x7f4e3409e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:46.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.242+0000 7f4e39a4a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e340b3430 0x7f4e34151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:46.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.242+0000 7f4e39a4a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e340d1440 0x7f4e340b88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:46.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.242+0000 7f4e39a4a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4e340b3b00 con 0x7f4e340b3430 2024-09-17T04:34:46.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.242+0000 7f4e39a4a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4e340b3980 con 0x7f4e340d1440 2024-09-17T04:34:46.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.242+0000 7f4e39a4a640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f4e34163290 con 0x7f4e3406f9f0 2024-09-17T04:34:46.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.242+0000 7f4e327fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e340b3430 0x7f4e34151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:46.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.242+0000 7f4e327fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e340b3430 0x7f4e34151650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39606/0 (socket says 172.21.15.37:39606) 2024-09-17T04:34:46.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.242+0000 7f4e327fc640 1 -- 172.21.15.37:0/3830807842 learned_addr learned my addr 172.21.15.37:0/3830807842 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:46.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.243+0000 7f4e337fe640 1 --2- 172.21.15.37:0/3830807842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e340d1440 0x7f4e340b88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:46.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.243+0000 7f4e32ffd640 1 --2- 172.21.15.37:0/3830807842 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4e3406f9f0 0x7f4e3409e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:46.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.243+0000 7f4e327fc640 1 -- 172.21.15.37:0/3830807842 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4e3406f9f0 msgr2=0x7f4e3409e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:46.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.243+0000 7f4e327fc640 1 --2- 172.21.15.37:0/3830807842 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4e3406f9f0 0x7f4e3409e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:46.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.243+0000 7f4e327fc640 1 -- 172.21.15.37:0/3830807842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e340d1440 msgr2=0x7f4e340b88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:46.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.243+0000 7f4e327fc640 1 --2- 172.21.15.37:0/3830807842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e340d1440 0x7f4e340b88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:46.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.243+0000 7f4e327fc640 1 -- 172.21.15.37:0/3830807842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e34151f30 con 0x7f4e340b3430 2024-09-17T04:34:46.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.243+0000 7f4e32ffd640 1 --2- 172.21.15.37:0/3830807842 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4e3406f9f0 0x7f4e3409e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:46.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.243+0000 7f4e327fc640 1 --2- 172.21.15.37:0/3830807842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e340b3430 0x7f4e34151650 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800c390 tx=0x7f4e2800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:46.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.243+0000 7f4e1bfff640 1 -- 172.21.15.37:0/3830807842 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e28017070 con 0x7f4e340b3430 2024-09-17T04:34:46.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.243+0000 7f4e1bfff640 1 -- 172.21.15.37:0/3830807842 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e280035f0 con 0x7f4e340b3430 2024-09-17T04:34:46.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.243+0000 7f4e39a4a640 1 -- 172.21.15.37:0/3830807842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e340cab50 con 0x7f4e340b3430 2024-09-17T04:34:46.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.243+0000 7f4e1bfff640 1 -- 172.21.15.37:0/3830807842 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e280039c0 con 0x7f4e340b3430 2024-09-17T04:34:46.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.243+0000 7f4e39a4a640 1 -- 172.21.15.37:0/3830807842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e34153770 con 0x7f4e340b3430 2024-09-17T04:34:46.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.244+0000 7f4e39a4a640 1 -- 172.21.15.37:0/3830807842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e340d34a0 con 0x7f4e340b3430 2024-09-17T04:34:46.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.245+0000 7f4e1bfff640 1 -- 172.21.15.37:0/3830807842 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4e2801a070 con 0x7f4e340b3430 2024-09-17T04:34:46.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.245+0000 7f4e1bfff640 1 --2- 172.21.15.37:0/3830807842 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4e20078b90 0x7f4e2007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:46.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.246+0000 7f4e1bfff640 1 -- 172.21.15.37:0/3830807842 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5029+0+0 (secure 0 0 0) 0x7f4e28099ce0 con 0x7f4e340b3430 2024-09-17T04:34:46.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.246+0000 7f4e32ffd640 1 --2- 172.21.15.37:0/3830807842 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4e20078b90 0x7f4e2007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:46.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.246+0000 7f4e32ffd640 1 --2- 172.21.15.37:0/3830807842 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4e20078b90 0x7f4e2007b0a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4e24033000 tx=0x7f4e24004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:46.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.250+0000 7f4e1bfff640 1 -- 172.21.15.37:0/3830807842 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4e340d1850 con 0x7f4e340b3430 2024-09-17T04:34:46.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.431+0000 7f4e39a4a640 1 -- 172.21.15.37:0/3830807842 --> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4e3406a150 con 0x7f4e20078b90 2024-09-17T04:34:46.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.432+0000 7f4e1bfff640 1 -- 172.21.15.37:0/3830807842 <== mgr.4100 v2:172.21.15.92:6800/864337835 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13481 (secure 0 0 0) 0x7f4e3406a150 con 0x7f4e20078b90 2024-09-17T04:34:46.432 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:46.432 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-17T04:34:46.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.434+0000 7f4e39a4a640 1 -- 172.21.15.37:0/3830807842 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4e20078b90 msgr2=0x7f4e2007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:46.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.434+0000 7f4e39a4a640 1 --2- 172.21.15.37:0/3830807842 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4e20078b90 0x7f4e2007b0a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4e24033000 tx=0x7f4e24004730 comp rx=0 tx=0).stop 2024-09-17T04:34:46.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.434+0000 7f4e39a4a640 1 -- 172.21.15.37:0/3830807842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e340b3430 msgr2=0x7f4e34151650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:46.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.434+0000 7f4e39a4a640 1 --2- 172.21.15.37:0/3830807842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e340b3430 0x7f4e34151650 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800c390 tx=0x7f4e2800c850 comp rx=0 tx=0).stop 2024-09-17T04:34:46.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.434+0000 7f4e39a4a640 1 -- 172.21.15.37:0/3830807842 shutdown_connections 2024-09-17T04:34:46.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.434+0000 7f4e39a4a640 1 --2- 172.21.15.37:0/3830807842 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4e20078b90 0x7f4e2007b0a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:46.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.434+0000 7f4e39a4a640 1 --2- 172.21.15.37:0/3830807842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e340d1440 0x7f4e340b88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:46.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.434+0000 7f4e39a4a640 1 --2- 172.21.15.37:0/3830807842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e340b3430 0x7f4e34151650 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:46.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.434+0000 7f4e39a4a640 1 --2- 172.21.15.37:0/3830807842 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4e3406f9f0 0x7f4e3409e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:46.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.434+0000 7f4e39a4a640 1 -- 172.21.15.37:0/3830807842 >> 172.21.15.37:0/3830807842 conn(0x7f4e3408bef0 msgr2=0x7f4e340b9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:46.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.434+0000 7f4e39a4a640 1 -- 172.21.15.37:0/3830807842 shutdown_connections 2024-09-17T04:34:46.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.434+0000 7f4e39a4a640 1 -- 172.21.15.37:0/3830807842 wait complete. 2024-09-17T04:34:46.445 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-09-17T04:34:44.466429+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":218,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237844,"kb_used_data":17440,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754736876,"statfs":{"total":773094113280,"available":772850561024,"internally_reserved":0,"allocated":17858560,"data_stored":15471022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6322167,"num_objects":-15,"num_object_clones":0,"num_object_copies":-30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-15,"num_whiteouts":0,"num_read":-53,"num_read_kb":-39,"num_write":-46,"num_write_kb":-6192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.913078"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":151,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T04:34:41.150326+0000","last_change":"2024-09-17T04:33:09.978742+0000","last_active":"2024-09-17T04:34:41.150326+0000","last_peered":"2024-09-17T04:34:41.150326+0000","last_clean":"2024-09-17T04:34:41.150326+0000","last_became_active":"2024-09-17T04:33:09.977496+0000","last_became_peered":"2024-09-17T04:33:09.977496+0000","last_unstale":"2024-09-17T04:34:41.150326+0000","last_undegraded":"2024-09-17T04:34:41.150326+0000","last_fullsized":"2024-09-17T04:34:41.150326+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:08.930340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:08.930340+0000","last_clean_scrub_stamp":"2024-09-17T04:33:08.930340+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:21:26.566102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542167,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30144,"kb_used_data":3116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341696,"statfs":{"total":96636764160,"available":96605896704,"internally_reserved":0,"allocated":3190784,"data_stored":2893354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542167,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29568,"kb_used_data":2540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342272,"statfs":{"total":96636764160,"available":96606486528,"internally_reserved":0,"allocated":2600960,"data_stored":2303568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542168,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":4584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":4694016,"data_stored":4400118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542167,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":536576,"data_stored":235657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542168,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":507904,"data_stored":206382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542167,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":540672,"data_stored":235679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542166,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":4584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":4694016,"data_stored":4400118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542167,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28100,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343740,"statfs":{"total":96636764160,"available":96607989760,"internally_reserved":0,"allocated":1093632,"data_stored":796146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:34:46.445 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T04:34:46.446 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:34:46.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.563+0000 7fc8ccd72640 1 -- 172.21.15.37:0/3392022899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8b0004800 msgr2=0x7fc8b0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:46.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.563+0000 7fc8ccd72640 1 --2- 172.21.15.37:0/3392022899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8b0004800 0x7fc8b0004c10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009b10 tx=0x7fc8b0005830 comp rx=0 tx=0).stop 2024-09-17T04:34:46.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.563+0000 7fc8c6d9d640 1 -- 172.21.15.37:0/3392022899 reap_dead start 2024-09-17T04:34:46.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.563+0000 7fc8ccd72640 1 -- 172.21.15.37:0/3392022899 shutdown_connections 2024-09-17T04:34:46.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.563+0000 7fc8ccd72640 1 -- 172.21.15.37:0/3392022899 >> 172.21.15.37:0/3392022899 conn(0x7fc8c808bef0 msgr2=0x7fc8c80d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:46.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.563+0000 7fc8ccd72640 1 -- 172.21.15.37:0/3392022899 shutdown_connections 2024-09-17T04:34:46.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.563+0000 7fc8ccd72640 1 -- 172.21.15.37:0/3392022899 wait complete. 2024-09-17T04:34:46.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.565+0000 7fc8ccd72640 1 Processor -- start 2024-09-17T04:34:46.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.565+0000 7fc8ccd72640 1 -- start start 2024-09-17T04:34:46.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.566+0000 7fc8ccd72640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8b0004800 0x7fc8c81f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:46.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.566+0000 7fc8ccd72640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc8c80c8b70 0x7fc8c815ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:46.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.566+0000 7fc8ccd72640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8c80cb6e0 0x7fc8c80cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:46.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.566+0000 7fc8ccd72640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c80d1610 con 0x7fc8c80cb6e0 2024-09-17T04:34:46.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.566+0000 7fc8ccd72640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c80b3b90 con 0x7fc8b0004800 2024-09-17T04:34:46.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.566+0000 7fc8ccd72640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fc8c80d1790 con 0x7fc8c80c8b70 2024-09-17T04:34:46.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.566+0000 7fc8c6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8c80cb6e0 0x7fc8c80cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:46.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.566+0000 7fc8c6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8c80cb6e0 0x7fc8c80cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39632/0 (socket says 172.21.15.37:39632) 2024-09-17T04:34:46.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.566+0000 7fc8c6d9d640 1 -- 172.21.15.37:0/4171366898 learned_addr learned my addr 172.21.15.37:0/4171366898 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:46.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.566+0000 7fc8c5d9b640 1 --2- 172.21.15.37:0/4171366898 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc8c80c8b70 0x7fc8c815ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:46.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.566+0000 7fc8c659c640 1 --2- 172.21.15.37:0/4171366898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8b0004800 0x7fc8c81f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:46.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.567+0000 7fc8c6d9d640 1 -- 172.21.15.37:0/4171366898 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc8c80c8b70 msgr2=0x7fc8c815ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:46.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.567+0000 7fc8c6d9d640 1 --2- 172.21.15.37:0/4171366898 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc8c80c8b70 0x7fc8c815ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:46.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.567+0000 7fc8c6d9d640 1 -- 172.21.15.37:0/4171366898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8b0004800 msgr2=0x7fc8c81f6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:46.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.567+0000 7fc8c6d9d640 1 --2- 172.21.15.37:0/4171366898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8b0004800 0x7fc8c81f6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:46.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.567+0000 7fc8c6d9d640 1 -- 172.21.15.37:0/4171366898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c80c8440 con 0x7fc8c80cb6e0 2024-09-17T04:34:46.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.567+0000 7fc8c659c640 1 --2- 172.21.15.37:0/4171366898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8b0004800 0x7fc8c81f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:46.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.567+0000 7fc8c6d9d640 1 --2- 172.21.15.37:0/4171366898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8c80cb6e0 0x7fc8c80cbaf0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0001eb0 tx=0x7fc8c000e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:46.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.567+0000 7fc8b6ffd640 1 -- 172.21.15.37:0/4171366898 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0011660 con 0x7fc8c80cb6e0 2024-09-17T04:34:46.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.567+0000 7fc8b6ffd640 1 -- 172.21.15.37:0/4171366898 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8c0017070 con 0x7fc8c80cb6e0 2024-09-17T04:34:46.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.567+0000 7fc8ccd72640 1 -- 172.21.15.37:0/4171366898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c80c8720 con 0x7fc8c80cb6e0 2024-09-17T04:34:46.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.567+0000 7fc8b6ffd640 1 -- 172.21.15.37:0/4171366898 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0011b90 con 0x7fc8c80cb6e0 2024-09-17T04:34:46.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.567+0000 7fc8ccd72640 1 -- 172.21.15.37:0/4171366898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c815efb0 con 0x7fc8c80cb6e0 2024-09-17T04:34:46.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.568+0000 7fc8ccd72640 1 -- 172.21.15.37:0/4171366898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8c80c81b0 con 0x7fc8c80cb6e0 2024-09-17T04:34:46.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.569+0000 7fc8b6ffd640 1 -- 172.21.15.37:0/4171366898 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc8c001c460 con 0x7fc8c80cb6e0 2024-09-17T04:34:46.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.569+0000 7fc8b6ffd640 1 --2- 172.21.15.37:0/4171366898 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc8a8078b90 0x7fc8a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:46.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.569+0000 7fc8b6ffd640 1 -- 172.21.15.37:0/4171366898 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5029+0+0 (secure 0 0 0) 0x7fc8c009acd0 con 0x7fc8c80cb6e0 2024-09-17T04:34:46.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.570+0000 7fc8c659c640 1 --2- 172.21.15.37:0/4171366898 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc8a8078b90 0x7fc8a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:46.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.570+0000 7fc8c659c640 1 --2- 172.21.15.37:0/4171366898 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc8a8078b90 0x7fc8a807b0a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8006cb0 tx=0x7fc8b8006de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:46.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.574+0000 7fc8b6ffd640 1 -- 172.21.15.37:0/4171366898 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc8c00633b0 con 0x7fc8c80cb6e0 2024-09-17T04:34:46.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.757+0000 7fc8ccd72640 1 -- 172.21.15.37:0/4171366898 --> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc8c80d0f00 con 0x7fc8a8078b90 2024-09-17T04:34:46.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.758+0000 7fc8b6ffd640 1 -- 172.21.15.37:0/4171366898 <== mgr.4100 v2:172.21.15.92:6800/864337835 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13481 (secure 0 0 0) 0x7fc8c80d0f00 con 0x7fc8a8078b90 2024-09-17T04:34:46.758 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:46.758 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-17T04:34:46.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.760+0000 7fc8ccd72640 1 -- 172.21.15.37:0/4171366898 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc8a8078b90 msgr2=0x7fc8a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:46.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.760+0000 7fc8ccd72640 1 --2- 172.21.15.37:0/4171366898 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc8a8078b90 0x7fc8a807b0a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8006cb0 tx=0x7fc8b8006de0 comp rx=0 tx=0).stop 2024-09-17T04:34:46.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.760+0000 7fc8ccd72640 1 -- 172.21.15.37:0/4171366898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8c80cb6e0 msgr2=0x7fc8c80cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:46.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.760+0000 7fc8ccd72640 1 --2- 172.21.15.37:0/4171366898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8c80cb6e0 0x7fc8c80cbaf0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0001eb0 tx=0x7fc8c000e6b0 comp rx=0 tx=0).stop 2024-09-17T04:34:46.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.760+0000 7fc8ccd72640 1 -- 172.21.15.37:0/4171366898 shutdown_connections 2024-09-17T04:34:46.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.760+0000 7fc8ccd72640 1 --2- 172.21.15.37:0/4171366898 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc8a8078b90 0x7fc8a807b0a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:46.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.760+0000 7fc8ccd72640 1 --2- 172.21.15.37:0/4171366898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8c80cb6e0 0x7fc8c80cbaf0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:46.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.760+0000 7fc8ccd72640 1 --2- 172.21.15.37:0/4171366898 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc8c80c8b70 0x7fc8c815ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:46.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.760+0000 7fc8ccd72640 1 --2- 172.21.15.37:0/4171366898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8b0004800 0x7fc8c81f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:46.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.760+0000 7fc8ccd72640 1 -- 172.21.15.37:0/4171366898 >> 172.21.15.37:0/4171366898 conn(0x7fc8c808bef0 msgr2=0x7fc8c80d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:46.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.760+0000 7fc8ccd72640 1 -- 172.21.15.37:0/4171366898 shutdown_connections 2024-09-17T04:34:46.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.760+0000 7fc8ccd72640 1 -- 172.21.15.37:0/4171366898 wait complete. 2024-09-17T04:34:46.771 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2024-09-17T04:34:46.466868+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":218,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237844,"kb_used_data":17440,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754736876,"statfs":{"total":773094113280,"available":772850561024,"internally_reserved":0,"allocated":17858560,"data_stored":15471022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6322167,"num_objects":-15,"num_object_clones":0,"num_object_copies":-30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-15,"num_whiteouts":0,"num_read":-53,"num_read_kb":-39,"num_write":-46,"num_write_kb":-6192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001388"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":151,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T04:34:41.150326+0000","last_change":"2024-09-17T04:33:09.978742+0000","last_active":"2024-09-17T04:34:41.150326+0000","last_peered":"2024-09-17T04:34:41.150326+0000","last_clean":"2024-09-17T04:34:41.150326+0000","last_became_active":"2024-09-17T04:33:09.977496+0000","last_became_peered":"2024-09-17T04:33:09.977496+0000","last_unstale":"2024-09-17T04:34:41.150326+0000","last_undegraded":"2024-09-17T04:34:41.150326+0000","last_fullsized":"2024-09-17T04:34:41.150326+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:08.930340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:08.930340+0000","last_clean_scrub_stamp":"2024-09-17T04:33:08.930340+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:21:26.566102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542167,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30144,"kb_used_data":3116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341696,"statfs":{"total":96636764160,"available":96605896704,"internally_reserved":0,"allocated":3190784,"data_stored":2893354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542167,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29568,"kb_used_data":2540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342272,"statfs":{"total":96636764160,"available":96606486528,"internally_reserved":0,"allocated":2600960,"data_stored":2303568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542168,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":4584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":4694016,"data_stored":4400118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542167,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":536576,"data_stored":235657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542168,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":507904,"data_stored":206382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542167,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":540672,"data_stored":235679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542166,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":4584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":4694016,"data_stored":4400118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542167,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28100,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343740,"statfs":{"total":96636764160,"available":96607989760,"internally_reserved":0,"allocated":1093632,"data_stored":796146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:34:46.771 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T04:34:46.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.888+0000 7f76a7d57640 1 -- 172.21.15.37:0/3584802479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76a00cd1b0 msgr2=0x7f76a00cd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:46.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.888+0000 7f76a7d57640 1 --2- 172.21.15.37:0/3584802479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76a00cd1b0 0x7f76a00cd5c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f76a0074c60 tx=0x7f769802aa30 comp rx=0 tx=0).stop 2024-09-17T04:34:46.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.888+0000 7f76a7d57640 1 -- 172.21.15.37:0/3584802479 shutdown_connections 2024-09-17T04:34:46.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.888+0000 7f76a7d57640 1 --2- 172.21.15.37:0/3584802479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76a00cd1b0 0x7f76a00cd5c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:46.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.888+0000 7f76a7d57640 1 --2- 172.21.15.37:0/3584802479 >> v2:172.21.15.92:3300/0 conn(0x7f76a0072a90 0x7f76a00cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:46.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.888+0000 7f76a7d57640 1 -- 172.21.15.37:0/3584802479 >> 172.21.15.37:0/3584802479 conn(0x7f76a00d2460 msgr2=0x7f76a00d2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:46.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.888+0000 7f76a7d57640 1 -- 172.21.15.37:0/3584802479 shutdown_connections 2024-09-17T04:34:46.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.888+0000 7f76a7d57640 1 -- 172.21.15.37:0/3584802479 wait complete. 2024-09-17T04:34:46.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.890+0000 7f76a7d57640 1 Processor -- start 2024-09-17T04:34:46.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.890+0000 7f76a7d57640 1 -- start start 2024-09-17T04:34:46.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.891+0000 7f76a7d57640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76a0072a90 0x7f76a014e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:46.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.891+0000 7f76a7d57640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76a00cd1b0 0x7f76a014fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:46.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.891+0000 7f76a7d57640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f76a00cf590 0x7f76a00cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:46.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.891+0000 7f76a7d57640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f76a00cdc80 con 0x7f76a0072a90 2024-09-17T04:34:46.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.891+0000 7f76a7d57640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f76a00cdb00 con 0x7f76a00cd1b0 2024-09-17T04:34:46.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.891+0000 7f76a7d57640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f76a0161100 con 0x7f76a00cf590 2024-09-17T04:34:46.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.892+0000 7f76a5af3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76a0072a90 0x7f76a014e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:46.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.892+0000 7f76a5af3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76a0072a90 0x7f76a014e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39656/0 (socket says 172.21.15.37:39656) 2024-09-17T04:34:46.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.892+0000 7f76a5af3640 1 -- 172.21.15.37:0/2325907272 learned_addr learned my addr 172.21.15.37:0/2325907272 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:46.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.892+0000 7f76a62f4640 1 --2- 172.21.15.37:0/2325907272 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f76a00cf590 0x7f76a00cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:46.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.892+0000 7f76a52f2640 1 --2- 172.21.15.37:0/2325907272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76a00cd1b0 0x7f76a014fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:46.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.892+0000 7f76a5af3640 1 -- 172.21.15.37:0/2325907272 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f76a00cf590 msgr2=0x7f76a00cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:46.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.892+0000 7f76a5af3640 1 --2- 172.21.15.37:0/2325907272 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f76a00cf590 0x7f76a00cba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:46.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.892+0000 7f76a5af3640 1 -- 172.21.15.37:0/2325907272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76a00cd1b0 msgr2=0x7f76a014fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:46.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.892+0000 7f76a5af3640 1 --2- 172.21.15.37:0/2325907272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76a00cd1b0 0x7f76a014fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:46.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.892+0000 7f76a5af3640 1 -- 172.21.15.37:0/2325907272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a014ede0 con 0x7f76a0072a90 2024-09-17T04:34:46.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.892+0000 7f76a5af3640 1 --2- 172.21.15.37:0/2325907272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76a0072a90 0x7f76a014e8a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f769c00c970 tx=0x7f769c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:46.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.892+0000 7f76967fc640 1 -- 172.21.15.37:0/2325907272 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c007b50 con 0x7f76a0072a90 2024-09-17T04:34:46.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.892+0000 7f76967fc640 1 -- 172.21.15.37:0/2325907272 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f769c007cf0 con 0x7f76a0072a90 2024-09-17T04:34:46.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.892+0000 7f76a7d57640 1 -- 172.21.15.37:0/2325907272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a015bc80 con 0x7f76a0072a90 2024-09-17T04:34:46.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.892+0000 7f76967fc640 1 -- 172.21.15.37:0/2325907272 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c005290 con 0x7f76a0072a90 2024-09-17T04:34:46.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.892+0000 7f76a7d57640 1 -- 172.21.15.37:0/2325907272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a015c240 con 0x7f76a0072a90 2024-09-17T04:34:46.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.893+0000 7f76a7d57640 1 -- 172.21.15.37:0/2325907272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a00cf050 con 0x7f76a0072a90 2024-09-17T04:34:46.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.894+0000 7f76967fc640 1 -- 172.21.15.37:0/2325907272 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f769c005430 con 0x7f76a0072a90 2024-09-17T04:34:46.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.895+0000 7f76967fc640 1 --2- 172.21.15.37:0/2325907272 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f7684078b90 0x7f768407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:46.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.895+0000 7f76967fc640 1 -- 172.21.15.37:0/2325907272 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5029+0+0 (secure 0 0 0) 0x7f769c09ad80 con 0x7f76a0072a90 2024-09-17T04:34:46.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.895+0000 7f76a52f2640 1 --2- 172.21.15.37:0/2325907272 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f7684078b90 0x7f768407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:46.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.896+0000 7f76a52f2640 1 --2- 172.21.15.37:0/2325907272 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f7684078b90 0x7f768407b0a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f76a015cfd0 tx=0x7f7690009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:46.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:46.899+0000 7f76967fc640 1 -- 172.21.15.37:0/2325907272 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f769c014070 con 0x7f76a0072a90 2024-09-17T04:34:47.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:47.088+0000 7f76a7d57640 1 -- 172.21.15.37:0/2325907272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76a006a150 con 0x7f76a0072a90 2024-09-17T04:34:47.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:47.088+0000 7f76967fc640 1 -- 172.21.15.37:0/2325907272 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+13102 (secure 0 0 0) 0x7f769c062ea0 con 0x7f76a0072a90 2024-09-17T04:34:47.088 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:47.088 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":45,"fsid":"a908a603-430c-4209-8294-0c3460b35f42","created":"2024-09-17T04:32:54.139221+0000","modified":"2024-09-17T04:34:44.032627+0000","last_up_change":"2024-09-17T04:33:06.922210+0000","last_in_change":"2024-09-17T04:32:57.754118+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T04:33:08.461907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6ea4b8d1-0d22-4dd9-b913-9df585ef24ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6809","nonce":2283587593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6811","nonce":2283587593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6815","nonce":2283587593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2283587593},{"type":"v1","addr":"172.21.15.37:6813","nonce":2283587593}]},"public_addr":"172.21.15.37:6809/2283587593","cluster_addr":"172.21.15.37:6811/2283587593","heartbeat_back_addr":"172.21.15.37:6815/2283587593","heartbeat_front_addr":"172.21.15.37:6813/2283587593","state":["exists","up"]},{"osd":1,"uuid":"ac745a33-e8ad-4083-9c18-71bf81982b76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6825","nonce":2657955935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6827","nonce":2657955935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6831","nonce":2657955935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":2657955935},{"type":"v1","addr":"172.21.15.37:6829","nonce":2657955935}]},"public_addr":"172.21.15.37:6825/2657955935","cluster_addr":"172.21.15.37:6827/2657955935","heartbeat_back_addr":"172.21.15.37:6831/2657955935","heartbeat_front_addr":"172.21.15.37:6829/2657955935","state":["exists","up"]},{"osd":2,"uuid":"3c63f3d8-132a-4735-b0bc-d64313a46503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6817","nonce":579720176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6819","nonce":579720176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6823","nonce":579720176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":579720176},{"type":"v1","addr":"172.21.15.37:6821","nonce":579720176}]},"public_addr":"172.21.15.37:6817/579720176","cluster_addr":"172.21.15.37:6819/579720176","heartbeat_back_addr":"172.21.15.37:6823/579720176","heartbeat_front_addr":"172.21.15.37:6821/579720176","state":["exists","up"]},{"osd":3,"uuid":"4f8c6ca2-ea1d-4610-9127-52c4dd8bc8e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6801","nonce":809975826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6803","nonce":809975826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6807","nonce":809975826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":809975826},{"type":"v1","addr":"172.21.15.37:6805","nonce":809975826}]},"public_addr":"172.21.15.37:6801/809975826","cluster_addr":"172.21.15.37:6803/809975826","heartbeat_back_addr":"172.21.15.37:6807/809975826","heartbeat_front_addr":"172.21.15.37:6805/809975826","state":["exists","up"]},{"osd":4,"uuid":"63d0c4ab-f115-47b4-9611-12a10153473a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6819","nonce":2509785752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6821","nonce":2509785752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6825","nonce":2509785752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2509785752},{"type":"v1","addr":"172.21.15.92:6823","nonce":2509785752}]},"public_addr":"172.21.15.92:6819/2509785752","cluster_addr":"172.21.15.92:6821/2509785752","heartbeat_back_addr":"172.21.15.92:6825/2509785752","heartbeat_front_addr":"172.21.15.92:6823/2509785752","state":["exists","up"]},{"osd":5,"uuid":"4c56c04d-0c07-4f76-9d75-6335121310ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6827","nonce":248263832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6829","nonce":248263832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6833","nonce":248263832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":248263832},{"type":"v1","addr":"172.21.15.92:6831","nonce":248263832}]},"public_addr":"172.21.15.92:6827/248263832","cluster_addr":"172.21.15.92:6829/248263832","heartbeat_back_addr":"172.21.15.92:6833/248263832","heartbeat_front_addr":"172.21.15.92:6831/248263832","state":["exists","up"]},{"osd":6,"uuid":"b30eae7c-096a-4d24-ac51-96b8b13ecaee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6811","nonce":4078365985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6813","nonce":4078365985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6817","nonce":4078365985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":4078365985},{"type":"v1","addr":"172.21.15.92:6815","nonce":4078365985}]},"public_addr":"172.21.15.92:6811/4078365985","cluster_addr":"172.21.15.92:6813/4078365985","heartbeat_back_addr":"172.21.15.92:6817/4078365985","heartbeat_front_addr":"172.21.15.92:6815/4078365985","state":["exists","up"]},{"osd":7,"uuid":"62f12540-d374-4a4c-9ff7-63b7304fe77f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6803","nonce":3444787223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6805","nonce":3444787223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6809","nonce":3444787223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3444787223},{"type":"v1","addr":"172.21.15.92:6807","nonce":3444787223}]},"public_addr":"172.21.15.92:6803/3444787223","cluster_addr":"172.21.15.92:6805/3444787223","heartbeat_back_addr":"172.21.15.92:6809/3444787223","heartbeat_front_addr":"172.21.15.92:6807/3444787223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.006420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.111135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.103325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.072656+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.200989+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:04.060455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.774773+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T04:33:03.865778+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/4051839052":"2024-09-18T04:34:41.017571+0000","172.21.15.37:6834/4051839052":"2024-09-18T04:34:41.017571+0000","172.21.15.37:6835/1358600605":"2024-09-18T04:34:15.699884+0000","172.21.15.37:6833/1358600605":"2024-09-18T04:34:15.699884+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T04:34:47.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:47.091+0000 7f76a7d57640 1 -- 172.21.15.37:0/2325907272 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f7684078b90 msgr2=0x7f768407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:47.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:47.091+0000 7f76a7d57640 1 --2- 172.21.15.37:0/2325907272 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f7684078b90 0x7f768407b0a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f76a015cfd0 tx=0x7f7690009060 comp rx=0 tx=0).stop 2024-09-17T04:34:47.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:47.091+0000 7f76a7d57640 1 -- 172.21.15.37:0/2325907272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76a0072a90 msgr2=0x7f76a014e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:47.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:47.091+0000 7f76a7d57640 1 --2- 172.21.15.37:0/2325907272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76a0072a90 0x7f76a014e8a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f769c00c970 tx=0x7f769c00ce30 comp rx=0 tx=0).stop 2024-09-17T04:34:47.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:47.091+0000 7f76a7d57640 1 -- 172.21.15.37:0/2325907272 shutdown_connections 2024-09-17T04:34:47.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:47.091+0000 7f76a7d57640 1 --2- 172.21.15.37:0/2325907272 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f7684078b90 0x7f768407b0a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:47.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:47.091+0000 7f76a7d57640 1 --2- 172.21.15.37:0/2325907272 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f76a00cf590 0x7f76a00cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:47.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:47.091+0000 7f76a7d57640 1 --2- 172.21.15.37:0/2325907272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76a00cd1b0 0x7f76a014fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:47.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:47.091+0000 7f76a7d57640 1 --2- 172.21.15.37:0/2325907272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76a0072a90 0x7f76a014e8a0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:47.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:47.091+0000 7f76a7d57640 1 -- 172.21.15.37:0/2325907272 >> 172.21.15.37:0/2325907272 conn(0x7f76a00d2460 msgr2=0x7f76a00b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:47.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:47.091+0000 7f76a7d57640 1 -- 172.21.15.37:0/2325907272 shutdown_connections 2024-09-17T04:34:47.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:47.091+0000 7f76a7d57640 1 -- 172.21.15.37:0/2325907272 wait complete. 2024-09-17T04:34:48.102 INFO:tasks.ceph:Scrubbing osd.0 2024-09-17T04:34:48.102 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-17T04:34:48.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.240+0000 7fa872b62640 1 -- 172.21.15.37:0/3330374241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa854004800 msgr2=0x7fa854004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:48.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.240+0000 7fa872b62640 1 --2- 172.21.15.37:0/3330374241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa854004800 0x7fa854004c10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa86000c470 tx=0x7fa854005830 comp rx=0 tx=0).stop 2024-09-17T04:34:48.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.240+0000 7fa8710ff640 1 -- 172.21.15.37:0/3330374241 reap_dead start 2024-09-17T04:34:48.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.240+0000 7fa872b62640 1 -- 172.21.15.37:0/3330374241 shutdown_connections 2024-09-17T04:34:48.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.240+0000 7fa872b62640 1 -- 172.21.15.37:0/3330374241 >> 172.21.15.37:0/3330374241 conn(0x7fa86c06d4a0 msgr2=0x7fa86c0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:48.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.240+0000 7fa872b62640 1 -- 172.21.15.37:0/3330374241 shutdown_connections 2024-09-17T04:34:48.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.240+0000 7fa872b62640 1 -- 172.21.15.37:0/3330374241 wait complete. 2024-09-17T04:34:48.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.242+0000 7fa872b62640 1 Processor -- start 2024-09-17T04:34:48.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.242+0000 7fa872b62640 1 -- start start 2024-09-17T04:34:48.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.243+0000 7fa872b62640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa854004800 0x7fa86c1f80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:48.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.243+0000 7fa872b62640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa86c1f8610 0x7fa86c15b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:48.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.243+0000 7fa872b62640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa86c0c97c0 0x7fa86c15bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:48.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.243+0000 7fa872b62640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c161040 con 0x7fa86c1f8610 2024-09-17T04:34:48.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.243+0000 7fa872b62640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c160ec0 con 0x7fa86c0c97c0 2024-09-17T04:34:48.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.243+0000 7fa872b62640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fa86c1611c0 con 0x7fa854004800 2024-09-17T04:34:48.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.243+0000 7fa86bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa86c1f8610 0x7fa86c15b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:48.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.243+0000 7fa86bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa86c1f8610 0x7fa86c15b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39666/0 (socket says 172.21.15.37:39666) 2024-09-17T04:34:48.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.243+0000 7fa86bfff640 1 -- 172.21.15.37:0/2339008472 learned_addr learned my addr 172.21.15.37:0/2339008472 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:48.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.244+0000 7fa8708fe640 1 --2- 172.21.15.37:0/2339008472 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa854004800 0x7fa86c1f80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:48.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.244+0000 7fa8710ff640 1 --2- 172.21.15.37:0/2339008472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa86c0c97c0 0x7fa86c15bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:48.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.244+0000 7fa86bfff640 1 -- 172.21.15.37:0/2339008472 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa854004800 msgr2=0x7fa86c1f80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:48.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.244+0000 7fa86bfff640 1 --2- 172.21.15.37:0/2339008472 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa854004800 0x7fa86c1f80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.244+0000 7fa86bfff640 1 -- 172.21.15.37:0/2339008472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa86c0c97c0 msgr2=0x7fa86c15bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:48.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.244+0000 7fa86bfff640 1 --2- 172.21.15.37:0/2339008472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa86c0c97c0 0x7fa86c15bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.244+0000 7fa86bfff640 1 -- 172.21.15.37:0/2339008472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa86c155db0 con 0x7fa86c1f8610 2024-09-17T04:34:48.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.244+0000 7fa8708fe640 1 --2- 172.21.15.37:0/2339008472 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa854004800 0x7fa86c1f80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:48.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.244+0000 7fa8710ff640 1 --2- 172.21.15.37:0/2339008472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa86c0c97c0 0x7fa86c15bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:48.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.244+0000 7fa86bfff640 1 --2- 172.21.15.37:0/2339008472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa86c1f8610 0x7fa86c15b780 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa86000c590 tx=0x7fa860006f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:48.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.244+0000 7fa8697fa640 1 -- 172.21.15.37:0/2339008472 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8600065f0 con 0x7fa86c1f8610 2024-09-17T04:34:48.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.244+0000 7fa8697fa640 1 -- 172.21.15.37:0/2339008472 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa86002b740 con 0x7fa86c1f8610 2024-09-17T04:34:48.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.244+0000 7fa872b62640 1 -- 172.21.15.37:0/2339008472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa86c1f8e00 con 0x7fa86c1f8610 2024-09-17T04:34:48.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.244+0000 7fa8697fa640 1 -- 172.21.15.37:0/2339008472 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86002bb90 con 0x7fa86c1f8610 2024-09-17T04:34:48.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.244+0000 7fa872b62640 1 -- 172.21.15.37:0/2339008472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa86c1f93c0 con 0x7fa86c1f8610 2024-09-17T04:34:48.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.245+0000 7fa872b62640 1 -- 172.21.15.37:0/2339008472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa86c0b48a0 con 0x7fa86c1f8610 2024-09-17T04:34:48.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.246+0000 7fa8697fa640 1 -- 172.21.15.37:0/2339008472 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa860046020 con 0x7fa86c1f8610 2024-09-17T04:34:48.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.247+0000 7fa8697fa640 1 --2- 172.21.15.37:0/2339008472 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa858078b90 0x7fa85807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:48.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.247+0000 7fa8697fa640 1 -- 172.21.15.37:0/2339008472 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5029+0+0 (secure 0 0 0) 0x7fa8600bb310 con 0x7fa86c1f8610 2024-09-17T04:34:48.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.247+0000 7fa8708fe640 1 --2- 172.21.15.37:0/2339008472 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa858078b90 0x7fa85807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:48.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.247+0000 7fa8697fa640 1 --2- 172.21.15.37:0/2339008472 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x7fa85807eb40 0x7fa858080f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:48.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.247+0000 7fa8697fa640 1 -- 172.21.15.37:0/2339008472 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa860004e70 con 0x7fa85807eb40 2024-09-17T04:34:48.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.247+0000 7fa8697fa640 1 -- 172.21.15.37:0/2339008472 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa8600bb6c0 con 0x7fa86c1f8610 2024-09-17T04:34:48.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.247+0000 7fa8710ff640 1 --2- 172.21.15.37:0/2339008472 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x7fa85807eb40 0x7fa858080f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:48.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.247+0000 7fa8708fe640 1 --2- 172.21.15.37:0/2339008472 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa858078b90 0x7fa85807b0a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa85c005eb0 tx=0x7fa85c005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:48.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.247+0000 7fa8710ff640 1 --2- 172.21.15.37:0/2339008472 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x7fa85807eb40 0x7fa858080f90 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:48.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.248+0000 7fa8697fa640 1 -- 172.21.15.37:0/2339008472 <== osd.0 v2:172.21.15.37:6808/2283587593 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa860004e70 con 0x7fa85807eb40 2024-09-17T04:34:48.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.261+0000 7fa872b62640 1 -- 172.21.15.37:0/2339008472 --> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa86c1f8bb0 con 0x7fa85807eb40 2024-09-17T04:34:48.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.261+0000 7fa8697fa640 1 -- 172.21.15.37:0/2339008472 <== osd.0 v2:172.21.15.37:6808/2283587593 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa86c1f8bb0 con 0x7fa85807eb40 2024-09-17T04:34:48.261 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T04:34:48.261 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T04:34:48.261 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T04:34:48.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.261+0000 7fa872b62640 1 -- 172.21.15.37:0/2339008472 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x7fa85807eb40 msgr2=0x7fa858080f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:48.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.261+0000 7fa872b62640 1 --2- 172.21.15.37:0/2339008472 >> [v2:172.21.15.37:6808/2283587593,v1:172.21.15.37:6809/2283587593] conn(0x7fa85807eb40 0x7fa858080f90 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.261+0000 7fa872b62640 1 -- 172.21.15.37:0/2339008472 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa858078b90 msgr2=0x7fa85807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:48.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.261+0000 7fa872b62640 1 --2- 172.21.15.37:0/2339008472 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fa858078b90 0x7fa85807b0a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa85c005eb0 tx=0x7fa85c005f90 comp rx=0 tx=0).stop 2024-09-17T04:34:48.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.261+0000 7fa872b62640 1 -- 172.21.15.37:0/2339008472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa86c1f8610 msgr2=0x7fa86c15b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:48.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.261+0000 7fa872b62640 1 --2- 172.21.15.37:0/2339008472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa86c1f8610 0x7fa86c15b780 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa86000c590 tx=0x7fa860006f60 comp rx=0 tx=0).stop 2024-09-17T04:34:48.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.261+0000 7fa8710ff640 1 -- 172.21.15.37:0/2339008472 reap_dead start 2024-09-17T04:34:48.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.262+0000 7fa872b62640 1 -- 172.21.15.37:0/2339008472 shutdown_connections 2024-09-17T04:34:48.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.262+0000 7fa872b62640 1 -- 172.21.15.37:0/2339008472 >> 172.21.15.37:0/2339008472 conn(0x7fa86c06d4a0 msgr2=0x7fa86c1602d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:48.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.262+0000 7fa872b62640 1 -- 172.21.15.37:0/2339008472 shutdown_connections 2024-09-17T04:34:48.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.262+0000 7fa872b62640 1 -- 172.21.15.37:0/2339008472 wait complete. 2024-09-17T04:34:48.272 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-17T04:34:48.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.391+0000 7fb70e1a1640 1 -- 172.21.15.37:0/3729976787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7080af7a0 msgr2=0x7fb7080b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:48.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.391+0000 7fb70e1a1640 1 --2- 172.21.15.37:0/3729976787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7080af7a0 0x7fb7080b4d60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8005fd0 tx=0x7fb6f802a530 comp rx=0 tx=0).stop 2024-09-17T04:34:48.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.391+0000 7fb70e1a1640 1 -- 172.21.15.37:0/3729976787 shutdown_connections 2024-09-17T04:34:48.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.391+0000 7fb70e1a1640 1 --2- 172.21.15.37:0/3729976787 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb708070010 0x7fb7080b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.391+0000 7fb70e1a1640 1 --2- 172.21.15.37:0/3729976787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7080af7a0 0x7fb7080b4d60 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.391+0000 7fb70e1a1640 1 -- 172.21.15.37:0/3729976787 >> 172.21.15.37:0/3729976787 conn(0x7fb70806d3f0 msgr2=0x7fb7080d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:48.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.391+0000 7fb70e1a1640 1 -- 172.21.15.37:0/3729976787 shutdown_connections 2024-09-17T04:34:48.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.391+0000 7fb70e1a1640 1 -- 172.21.15.37:0/3729976787 wait complete. 2024-09-17T04:34:48.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.392+0000 7fb70e1a1640 1 Processor -- start 2024-09-17T04:34:48.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.393+0000 7fb70e1a1640 1 -- start start 2024-09-17T04:34:48.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.394+0000 7fb70e1a1640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb708070010 0x7fb70815e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:48.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.394+0000 7fb70e1a1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7080af7a0 0x7fb7081f2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:48.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.394+0000 7fb70e1a1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7080ca890 0x7fb708155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:48.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.394+0000 7fb70e1a1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb7080b3bd0 con 0x7fb7080af7a0 2024-09-17T04:34:48.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.394+0000 7fb70e1a1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb7080b3a50 con 0x7fb7080ca890 2024-09-17T04:34:48.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.394+0000 7fb70e1a1640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fb7080b3d50 con 0x7fb708070010 2024-09-17T04:34:48.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.394+0000 7fb706ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7080af7a0 0x7fb7081f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:48.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.394+0000 7fb706ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7080af7a0 0x7fb7081f2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39688/0 (socket says 172.21.15.37:39688) 2024-09-17T04:34:48.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.394+0000 7fb706ffd640 1 -- 172.21.15.37:0/3408163918 learned_addr learned my addr 172.21.15.37:0/3408163918 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:48.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.394+0000 7fb707fff640 1 --2- 172.21.15.37:0/3408163918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7080ca890 0x7fb708155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:48.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.394+0000 7fb7077fe640 1 --2- 172.21.15.37:0/3408163918 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb708070010 0x7fb70815e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:48.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.394+0000 7fb706ffd640 1 -- 172.21.15.37:0/3408163918 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb708070010 msgr2=0x7fb70815e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:48.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.394+0000 7fb706ffd640 1 --2- 172.21.15.37:0/3408163918 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb708070010 0x7fb70815e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.394+0000 7fb706ffd640 1 -- 172.21.15.37:0/3408163918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7080ca890 msgr2=0x7fb708155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:48.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.394+0000 7fb706ffd640 1 --2- 172.21.15.37:0/3408163918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7080ca890 0x7fb708155c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.394+0000 7fb706ffd640 1 -- 172.21.15.37:0/3408163918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70815cf60 con 0x7fb7080af7a0 2024-09-17T04:34:48.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.394+0000 7fb707fff640 1 --2- 172.21.15.37:0/3408163918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7080ca890 0x7fb708155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:48.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.395+0000 7fb706ffd640 1 --2- 172.21.15.37:0/3408163918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7080af7a0 0x7fb7081f2500 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00b3f0 tx=0x7fb6fc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:48.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.395+0000 7fb6effff640 1 -- 172.21.15.37:0/3408163918 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc015020 con 0x7fb7080af7a0 2024-09-17T04:34:48.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.395+0000 7fb6effff640 1 -- 172.21.15.37:0/3408163918 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6fc011920 con 0x7fb7080af7a0 2024-09-17T04:34:48.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.395+0000 7fb70e1a1640 1 -- 172.21.15.37:0/3408163918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb70815d3d0 con 0x7fb7080af7a0 2024-09-17T04:34:48.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.395+0000 7fb6effff640 1 -- 172.21.15.37:0/3408163918 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc011d70 con 0x7fb7080af7a0 2024-09-17T04:34:48.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.395+0000 7fb70e1a1640 1 -- 172.21.15.37:0/3408163918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb70815d990 con 0x7fb7080af7a0 2024-09-17T04:34:48.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.396+0000 7fb70e1a1640 1 -- 172.21.15.37:0/3408163918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7080b4280 con 0x7fb7080af7a0 2024-09-17T04:34:48.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.397+0000 7fb6effff640 1 -- 172.21.15.37:0/3408163918 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb6fc018020 con 0x7fb7080af7a0 2024-09-17T04:34:48.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.397+0000 7fb6effff640 1 --2- 172.21.15.37:0/3408163918 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb6f4078bc0 0x7fb6f407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:48.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.397+0000 7fb6effff640 1 -- 172.21.15.37:0/3408163918 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5029+0+0 (secure 0 0 0) 0x7fb6fc09a950 con 0x7fb7080af7a0 2024-09-17T04:34:48.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.397+0000 7fb7077fe640 1 --2- 172.21.15.37:0/3408163918 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb6f4078bc0 0x7fb6f407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:48.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.398+0000 7fb7077fe640 1 --2- 172.21.15.37:0/3408163918 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb6f4078bc0 0x7fb6f407b0d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8033000 tx=0x7fb6f8004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:48.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.402+0000 7fb6effff640 1 -- 172.21.15.37:0/3408163918 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb6fc01f020 con 0x7fb7080af7a0 2024-09-17T04:34:48.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.592+0000 7fb70e1a1640 1 -- 172.21.15.37:0/3408163918 --> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fb70815d130 con 0x7fb6f4078bc0 2024-09-17T04:34:48.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.593+0000 7fb6effff640 1 -- 172.21.15.37:0/3408163918 <== mgr.4100 v2:172.21.15.92:6800/864337835 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb70815d130 con 0x7fb6f4078bc0 2024-09-17T04:34:48.593 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 0 to deep-scrub 2024-09-17T04:34:48.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.595+0000 7fb70e1a1640 1 -- 172.21.15.37:0/3408163918 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb6f4078bc0 msgr2=0x7fb6f407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:48.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.595+0000 7fb70e1a1640 1 --2- 172.21.15.37:0/3408163918 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb6f4078bc0 0x7fb6f407b0d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8033000 tx=0x7fb6f8004730 comp rx=0 tx=0).stop 2024-09-17T04:34:48.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.595+0000 7fb70e1a1640 1 -- 172.21.15.37:0/3408163918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7080af7a0 msgr2=0x7fb7081f2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:48.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.595+0000 7fb70e1a1640 1 --2- 172.21.15.37:0/3408163918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7080af7a0 0x7fb7081f2500 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00b3f0 tx=0x7fb6fc00b8b0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.595+0000 7fb70e1a1640 1 -- 172.21.15.37:0/3408163918 shutdown_connections 2024-09-17T04:34:48.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.595+0000 7fb70e1a1640 1 --2- 172.21.15.37:0/3408163918 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb6f4078bc0 0x7fb6f407b0d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.595+0000 7fb70e1a1640 1 --2- 172.21.15.37:0/3408163918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7080ca890 0x7fb708155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.595+0000 7fb70e1a1640 1 --2- 172.21.15.37:0/3408163918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7080af7a0 0x7fb7081f2500 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.595+0000 7fb70e1a1640 1 --2- 172.21.15.37:0/3408163918 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb708070010 0x7fb70815e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.595+0000 7fb70e1a1640 1 -- 172.21.15.37:0/3408163918 >> 172.21.15.37:0/3408163918 conn(0x7fb70806d3f0 msgr2=0x7fb708159b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:48.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.595+0000 7fb70e1a1640 1 -- 172.21.15.37:0/3408163918 shutdown_connections 2024-09-17T04:34:48.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.595+0000 7fb70e1a1640 1 -- 172.21.15.37:0/3408163918 wait complete. 2024-09-17T04:34:48.606 INFO:tasks.ceph:Scrubbing osd.1 2024-09-17T04:34:48.606 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-17T04:34:48.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.726+0000 7fc63196e640 1 -- 172.21.15.37:0/934603275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc62c0b34c0 msgr2=0x7fc62c161ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:48.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.726+0000 7fc63196e640 1 --2- 172.21.15.37:0/934603275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc62c0b34c0 0x7fc62c161ff0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc61c005fd0 tx=0x7fc61c02a5d0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.726+0000 7fc63196e640 1 -- 172.21.15.37:0/934603275 shutdown_connections 2024-09-17T04:34:48.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.726+0000 7fc63196e640 1 --2- 172.21.15.37:0/934603275 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc62c0b6750 0x7fc62c0d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.726+0000 7fc63196e640 1 --2- 172.21.15.37:0/934603275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc62c0b34c0 0x7fc62c161ff0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.726+0000 7fc63196e640 1 -- 172.21.15.37:0/934603275 >> 172.21.15.37:0/934603275 conn(0x7fc62c08bef0 msgr2=0x7fc62c0d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:48.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.727+0000 7fc63196e640 1 -- 172.21.15.37:0/934603275 shutdown_connections 2024-09-17T04:34:48.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.727+0000 7fc63196e640 1 -- 172.21.15.37:0/934603275 wait complete. 2024-09-17T04:34:48.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.728+0000 7fc63196e640 1 Processor -- start 2024-09-17T04:34:48.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.729+0000 7fc63196e640 1 -- start start 2024-09-17T04:34:48.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.730+0000 7fc63196e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc62c0b3a10 0x7fc62c09ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:48.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.730+0000 7fc63196e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc62c1f8d90 0x7fc62c1f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:48.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.730+0000 7fc63196e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc62c0cd7d0 0x7fc62c0cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:48.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.730+0000 7fc63196e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c0d2e30 con 0x7fc62c0b3a10 2024-09-17T04:34:48.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.730+0000 7fc63196e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c0d2cb0 con 0x7fc62c0cd7d0 2024-09-17T04:34:48.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.730+0000 7fc63196e640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fc62c0d2fb0 con 0x7fc62c1f8d90 2024-09-17T04:34:48.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.730+0000 7fc62affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc62c0b3a10 0x7fc62c09ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:48.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.730+0000 7fc62affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc62c0b3a10 0x7fc62c09ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39706/0 (socket says 172.21.15.37:39706) 2024-09-17T04:34:48.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.730+0000 7fc62affd640 1 -- 172.21.15.37:0/2474832608 learned_addr learned my addr 172.21.15.37:0/2474832608 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:48.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.730+0000 7fc62b7fe640 1 --2- 172.21.15.37:0/2474832608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc62c0cd7d0 0x7fc62c0cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:48.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.730+0000 7fc62a7fc640 1 --2- 172.21.15.37:0/2474832608 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc62c1f8d90 0x7fc62c1f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:48.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.730+0000 7fc62affd640 1 -- 172.21.15.37:0/2474832608 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc62c1f8d90 msgr2=0x7fc62c1f91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:48.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.730+0000 7fc62affd640 1 --2- 172.21.15.37:0/2474832608 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc62c1f8d90 0x7fc62c1f91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.730+0000 7fc62affd640 1 -- 172.21.15.37:0/2474832608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc62c0cd7d0 msgr2=0x7fc62c0cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:48.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.730+0000 7fc62affd640 1 --2- 172.21.15.37:0/2474832608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc62c0cd7d0 0x7fc62c0cdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.730+0000 7fc62affd640 1 -- 172.21.15.37:0/2474832608 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62c0ce120 con 0x7fc62c0b3a10 2024-09-17T04:34:48.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.730+0000 7fc62b7fe640 1 --2- 172.21.15.37:0/2474832608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc62c0cd7d0 0x7fc62c0cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:48.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.731+0000 7fc62affd640 1 --2- 172.21.15.37:0/2474832608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc62c0b3a10 0x7fc62c09ea50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc61c034040 tx=0x7fc61c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:48.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.731+0000 7fc613fff640 1 -- 172.21.15.37:0/2474832608 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c03e030 con 0x7fc62c0b3a10 2024-09-17T04:34:48.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.731+0000 7fc613fff640 1 -- 172.21.15.37:0/2474832608 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc61c003ae0 con 0x7fc62c0b3a10 2024-09-17T04:34:48.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.731+0000 7fc63196e640 1 -- 172.21.15.37:0/2474832608 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc62c0b4070 con 0x7fc62c0b3a10 2024-09-17T04:34:48.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.731+0000 7fc613fff640 1 -- 172.21.15.37:0/2474832608 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c035040 con 0x7fc62c0b3a10 2024-09-17T04:34:48.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.731+0000 7fc63196e640 1 -- 172.21.15.37:0/2474832608 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc62c0b4630 con 0x7fc62c0b3a10 2024-09-17T04:34:48.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.732+0000 7fc63196e640 1 -- 172.21.15.37:0/2474832608 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc62c0b3e20 con 0x7fc62c0b3a10 2024-09-17T04:34:48.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.733+0000 7fc613fff640 1 -- 172.21.15.37:0/2474832608 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc61c041030 con 0x7fc62c0b3a10 2024-09-17T04:34:48.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.733+0000 7fc613fff640 1 --2- 172.21.15.37:0/2474832608 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc618078b90 0x7fc61807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:48.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.733+0000 7fc613fff640 1 -- 172.21.15.37:0/2474832608 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5029+0+0 (secure 0 0 0) 0x7fc61c0bb620 con 0x7fc62c0b3a10 2024-09-17T04:34:48.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.733+0000 7fc613fff640 1 --2- 172.21.15.37:0/2474832608 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x7fc61807eb40 0x7fc618080f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:48.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.733+0000 7fc613fff640 1 -- 172.21.15.37:0/2474832608 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc61c005310 con 0x7fc61807eb40 2024-09-17T04:34:48.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.733+0000 7fc613fff640 1 -- 172.21.15.37:0/2474832608 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fc61c0bbab0 con 0x7fc62c0b3a10 2024-09-17T04:34:48.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.733+0000 7fc62a7fc640 1 --2- 172.21.15.37:0/2474832608 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc618078b90 0x7fc61807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:48.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.733+0000 7fc62b7fe640 1 --2- 172.21.15.37:0/2474832608 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x7fc61807eb40 0x7fc618080f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:48.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.734+0000 7fc62b7fe640 1 --2- 172.21.15.37:0/2474832608 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x7fc61807eb40 0x7fc618080f90 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:48.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.734+0000 7fc62a7fc640 1 --2- 172.21.15.37:0/2474832608 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc618078b90 0x7fc61807b0a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0b9ad0 tx=0x7fc620009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:48.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.734+0000 7fc613fff640 1 -- 172.21.15.37:0/2474832608 <== osd.1 v2:172.21.15.37:6824/2657955935 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc61c005310 con 0x7fc61807eb40 2024-09-17T04:34:48.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.747+0000 7fc63196e640 1 -- 172.21.15.37:0/2474832608 --> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc62c0d34d0 con 0x7fc61807eb40 2024-09-17T04:34:48.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.747+0000 7fc613fff640 1 -- 172.21.15.37:0/2474832608 <== osd.1 v2:172.21.15.37:6824/2657955935 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc62c0d34d0 con 0x7fc61807eb40 2024-09-17T04:34:48.747 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T04:34:48.748 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T04:34:48.748 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T04:34:48.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.748+0000 7fc63196e640 1 -- 172.21.15.37:0/2474832608 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x7fc61807eb40 msgr2=0x7fc618080f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:48.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.748+0000 7fc63196e640 1 --2- 172.21.15.37:0/2474832608 >> [v2:172.21.15.37:6824/2657955935,v1:172.21.15.37:6825/2657955935] conn(0x7fc61807eb40 0x7fc618080f90 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.748+0000 7fc63196e640 1 -- 172.21.15.37:0/2474832608 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc618078b90 msgr2=0x7fc61807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:48.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.748+0000 7fc63196e640 1 --2- 172.21.15.37:0/2474832608 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fc618078b90 0x7fc61807b0a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0b9ad0 tx=0x7fc620009040 comp rx=0 tx=0).stop 2024-09-17T04:34:48.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.748+0000 7fc63196e640 1 -- 172.21.15.37:0/2474832608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc62c0b3a10 msgr2=0x7fc62c09ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:48.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.748+0000 7fc63196e640 1 --2- 172.21.15.37:0/2474832608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc62c0b3a10 0x7fc62c09ea50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc61c034040 tx=0x7fc61c0035f0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.748+0000 7fc62b7fe640 1 -- 172.21.15.37:0/2474832608 reap_dead start 2024-09-17T04:34:48.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.748+0000 7fc63196e640 1 -- 172.21.15.37:0/2474832608 shutdown_connections 2024-09-17T04:34:48.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.748+0000 7fc63196e640 1 -- 172.21.15.37:0/2474832608 >> 172.21.15.37:0/2474832608 conn(0x7fc62c08bef0 msgr2=0x7fc62c15f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:48.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.748+0000 7fc63196e640 1 -- 172.21.15.37:0/2474832608 shutdown_connections 2024-09-17T04:34:48.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.748+0000 7fc63196e640 1 -- 172.21.15.37:0/2474832608 wait complete. 2024-09-17T04:34:48.759 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-17T04:34:48.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.876+0000 7fadfd0bd640 1 -- 172.21.15.37:0/3132055739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadf806f9f0 msgr2=0x7fadf80d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:48.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.876+0000 7fadfd0bd640 1 --2- 172.21.15.37:0/3132055739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadf806f9f0 0x7fadf80d0f00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fadf80cb6b0 tx=0x7fade802a530 comp rx=0 tx=0).stop 2024-09-17T04:34:48.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.876+0000 7fadfd0bd640 1 -- 172.21.15.37:0/3132055739 shutdown_connections 2024-09-17T04:34:48.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.876+0000 7fadfd0bd640 1 --2- 172.21.15.37:0/3132055739 >> v2:172.21.15.92:3300/0 conn(0x7fadf80b3430 0x7fadf80d1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.876+0000 7fadfd0bd640 1 --2- 172.21.15.37:0/3132055739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadf806f9f0 0x7fadf80d0f00 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.876+0000 7fadfd0bd640 1 -- 172.21.15.37:0/3132055739 >> 172.21.15.37:0/3132055739 conn(0x7fadf808bef0 msgr2=0x7fadf80d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:48.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.876+0000 7fadfd0bd640 1 -- 172.21.15.37:0/3132055739 shutdown_connections 2024-09-17T04:34:48.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.876+0000 7fadfd0bd640 1 -- 172.21.15.37:0/3132055739 wait complete. 2024-09-17T04:34:48.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.878+0000 7fadfd0bd640 1 Processor -- start 2024-09-17T04:34:48.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.878+0000 7fadfd0bd640 1 -- start start 2024-09-17T04:34:48.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.879+0000 7fadfd0bd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadf806f9f0 0x7fadf81f9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:48.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.879+0000 7fadfd0bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadf80b3430 0x7fadf81f6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:48.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.879+0000 7fadfd0bd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fadf8162640 0x7fadf80a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.879+0000 7fadfd0bd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fadf80b3b00 con 0x7fadf80b3430 2024-09-17T04:34:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.879+0000 7fadfd0bd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fadf80b3980 con 0x7fadf806f9f0 2024-09-17T04:34:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.879+0000 7fadfd0bd640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fadf80d33c0 con 0x7fadf8162640 2024-09-17T04:34:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.879+0000 7fadf659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadf80b3430 0x7fadf81f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.879+0000 7fadf659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadf80b3430 0x7fadf81f6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39730/0 (socket says 172.21.15.37:39730) 2024-09-17T04:34:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.879+0000 7fadf659c640 1 -- 172.21.15.37:0/3672653898 learned_addr learned my addr 172.21.15.37:0/3672653898 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.880+0000 7fadf6d9d640 1 --2- 172.21.15.37:0/3672653898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadf806f9f0 0x7fadf81f9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.880+0000 7fadf759e640 1 --2- 172.21.15.37:0/3672653898 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fadf8162640 0x7fadf80a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.880+0000 7fadf659c640 1 -- 172.21.15.37:0/3672653898 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fadf8162640 msgr2=0x7fadf80a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.880+0000 7fadf659c640 1 --2- 172.21.15.37:0/3672653898 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fadf8162640 0x7fadf80a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.880+0000 7fadf659c640 1 -- 172.21.15.37:0/3672653898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadf806f9f0 msgr2=0x7fadf81f9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.880+0000 7fadf659c640 1 --2- 172.21.15.37:0/3672653898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadf806f9f0 0x7fadf81f9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.880+0000 7fadf659c640 1 -- 172.21.15.37:0/3672653898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf81f9760 con 0x7fadf80b3430 2024-09-17T04:34:48.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.880+0000 7fadf6d9d640 1 --2- 172.21.15.37:0/3672653898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadf806f9f0 0x7fadf81f9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:48.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.880+0000 7fadf759e640 1 --2- 172.21.15.37:0/3672653898 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fadf8162640 0x7fadf80a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:48.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.880+0000 7fadf659c640 1 --2- 172.21.15.37:0/3672653898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadf80b3430 0x7fadf81f6660 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fadec00ea20 tx=0x7fadec00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:48.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.880+0000 7faddf7fe640 1 -- 172.21.15.37:0/3672653898 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec00cc60 con 0x7fadf80b3430 2024-09-17T04:34:48.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.880+0000 7faddf7fe640 1 -- 172.21.15.37:0/3672653898 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fadec010430 con 0x7fadf80b3430 2024-09-17T04:34:48.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.880+0000 7fadfd0bd640 1 -- 172.21.15.37:0/3672653898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf81f6ba0 con 0x7fadf80b3430 2024-09-17T04:34:48.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.880+0000 7faddf7fe640 1 -- 172.21.15.37:0/3672653898 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec010840 con 0x7fadf80b3430 2024-09-17T04:34:48.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.880+0000 7fadfd0bd640 1 -- 172.21.15.37:0/3672653898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf81f9f50 con 0x7fadf80b3430 2024-09-17T04:34:48.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.881+0000 7fadfd0bd640 1 -- 172.21.15.37:0/3672653898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadf80c7ce0 con 0x7fadf80b3430 2024-09-17T04:34:48.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.882+0000 7faddf7fe640 1 -- 172.21.15.37:0/3672653898 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fadec0109e0 con 0x7fadf80b3430 2024-09-17T04:34:48.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.883+0000 7faddf7fe640 1 --2- 172.21.15.37:0/3672653898 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fade4078bc0 0x7fade407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:48.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.883+0000 7faddf7fe640 1 -- 172.21.15.37:0/3672653898 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5029+0+0 (secure 0 0 0) 0x7fadec09b610 con 0x7fadf80b3430 2024-09-17T04:34:48.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.883+0000 7fadf6d9d640 1 --2- 172.21.15.37:0/3672653898 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fade4078bc0 0x7fade407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:48.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.884+0000 7fadf6d9d640 1 --2- 172.21.15.37:0/3672653898 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fade4078bc0 0x7fade407b0d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fade8034000 tx=0x7fade8004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:48.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:48.887+0000 7faddf7fe640 1 -- 172.21.15.37:0/3672653898 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fadf80c7ce0 con 0x7fadf80b3430 2024-09-17T04:34:49.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.073+0000 7fadfd0bd640 1 -- 172.21.15.37:0/3672653898 --> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fadf81f9a00 con 0x7fade4078bc0 2024-09-17T04:34:49.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.074+0000 7faddf7fe640 1 -- 172.21.15.37:0/3672653898 <== mgr.4100 v2:172.21.15.92:6800/864337835 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fadf81f9a00 con 0x7fade4078bc0 2024-09-17T04:34:49.073 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 1 to deep-scrub 2024-09-17T04:34:49.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.076+0000 7fadfd0bd640 1 -- 172.21.15.37:0/3672653898 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fade4078bc0 msgr2=0x7fade407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:49.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.076+0000 7fadfd0bd640 1 --2- 172.21.15.37:0/3672653898 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fade4078bc0 0x7fade407b0d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fade8034000 tx=0x7fade8004730 comp rx=0 tx=0).stop 2024-09-17T04:34:49.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.076+0000 7fadfd0bd640 1 -- 172.21.15.37:0/3672653898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadf80b3430 msgr2=0x7fadf81f6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:49.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.076+0000 7fadfd0bd640 1 --2- 172.21.15.37:0/3672653898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadf80b3430 0x7fadf81f6660 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fadec00ea20 tx=0x7fadec00eee0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.076+0000 7fadfd0bd640 1 -- 172.21.15.37:0/3672653898 shutdown_connections 2024-09-17T04:34:49.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.076+0000 7fadfd0bd640 1 --2- 172.21.15.37:0/3672653898 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fade4078bc0 0x7fade407b0d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.076+0000 7fadfd0bd640 1 --2- 172.21.15.37:0/3672653898 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fadf8162640 0x7fadf80a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.076+0000 7fadfd0bd640 1 --2- 172.21.15.37:0/3672653898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadf80b3430 0x7fadf81f6660 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.076+0000 7fadfd0bd640 1 --2- 172.21.15.37:0/3672653898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadf806f9f0 0x7fadf81f9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.076+0000 7fadfd0bd640 1 -- 172.21.15.37:0/3672653898 >> 172.21.15.37:0/3672653898 conn(0x7fadf808bef0 msgr2=0x7fadf806df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:49.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.076+0000 7fadfd0bd640 1 -- 172.21.15.37:0/3672653898 shutdown_connections 2024-09-17T04:34:49.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.076+0000 7fadfd0bd640 1 -- 172.21.15.37:0/3672653898 wait complete. 2024-09-17T04:34:49.089 INFO:tasks.ceph:Scrubbing osd.2 2024-09-17T04:34:49.089 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-17T04:34:49.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.210+0000 7f3e7a7a8640 1 -- 172.21.15.37:0/476071061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e740d31c0 msgr2=0x7f3e7406e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:49.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.210+0000 7f3e7a7a8640 1 --2- 172.21.15.37:0/476071061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e740d31c0 0x7f3e7406e8f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3e740d1ef0 tx=0x7f3e6c02aac0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.210+0000 7f3e7a7a8640 1 -- 172.21.15.37:0/476071061 shutdown_connections 2024-09-17T04:34:49.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.210+0000 7f3e7a7a8640 1 --2- 172.21.15.37:0/476071061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e740d31c0 0x7f3e7406e8f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.210+0000 7f3e7a7a8640 1 --2- 172.21.15.37:0/476071061 >> v2:172.21.15.92:3300/0 conn(0x7f3e740d16d0 0x7f3e740d2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.210+0000 7f3e7a7a8640 1 -- 172.21.15.37:0/476071061 >> 172.21.15.37:0/476071061 conn(0x7f3e7408bef0 msgr2=0x7f3e740d45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:49.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.210+0000 7f3e7a7a8640 1 -- 172.21.15.37:0/476071061 shutdown_connections 2024-09-17T04:34:49.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.210+0000 7f3e7a7a8640 1 -- 172.21.15.37:0/476071061 wait complete. 2024-09-17T04:34:49.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.212+0000 7f3e7a7a8640 1 Processor -- start 2024-09-17T04:34:49.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.213+0000 7f3e7a7a8640 1 -- start start 2024-09-17T04:34:49.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.214+0000 7f3e7a7a8640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3e740b3c00 0x7f3e740b68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:49.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.214+0000 7f3e7a7a8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e740d31c0 0x7f3e741f2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:49.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.214+0000 7f3e7a7a8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e741f8a10 0x7f3e741f8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:49.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.214+0000 7f3e7a7a8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7406f410 con 0x7f3e741f8a10 2024-09-17T04:34:49.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.214+0000 7f3e7a7a8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7406f290 con 0x7f3e740d31c0 2024-09-17T04:34:49.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.214+0000 7f3e7a7a8640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f3e7406f590 con 0x7f3e740b3c00 2024-09-17T04:34:49.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.214+0000 7f3e78d45640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e741f8a10 0x7f3e741f8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:49.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.214+0000 7f3e78d45640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e741f8a10 0x7f3e741f8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39742/0 (socket says 172.21.15.37:39742) 2024-09-17T04:34:49.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.214+0000 7f3e78d45640 1 -- 172.21.15.37:0/4034226485 learned_addr learned my addr 172.21.15.37:0/4034226485 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:49.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.214+0000 7f3e737fe640 1 --2- 172.21.15.37:0/4034226485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e740d31c0 0x7f3e741f2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:49.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.214+0000 7f3e73fff640 1 --2- 172.21.15.37:0/4034226485 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3e740b3c00 0x7f3e740b68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:49.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.214+0000 7f3e78d45640 1 -- 172.21.15.37:0/4034226485 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3e740b3c00 msgr2=0x7f3e740b68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:49.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.214+0000 7f3e78d45640 1 --2- 172.21.15.37:0/4034226485 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3e740b3c00 0x7f3e740b68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.214+0000 7f3e78d45640 1 -- 172.21.15.37:0/4034226485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e740d31c0 msgr2=0x7f3e741f2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:49.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.214+0000 7f3e78d45640 1 --2- 172.21.15.37:0/4034226485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e740d31c0 0x7f3e741f2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.214+0000 7f3e78d45640 1 -- 172.21.15.37:0/4034226485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e740b6e30 con 0x7f3e741f8a10 2024-09-17T04:34:49.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.214+0000 7f3e73fff640 1 --2- 172.21.15.37:0/4034226485 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3e740b3c00 0x7f3e740b68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:49.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.214+0000 7f3e78d45640 1 --2- 172.21.15.37:0/4034226485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e741f8a10 0x7f3e741f8e20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c02b0b0 tx=0x7f3e6c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:49.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.215+0000 7f3e70ff9640 1 -- 172.21.15.37:0/4034226485 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c004250 con 0x7f3e741f8a10 2024-09-17T04:34:49.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.215+0000 7f3e70ff9640 1 -- 172.21.15.37:0/4034226485 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e6c0043f0 con 0x7f3e741f8a10 2024-09-17T04:34:49.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.215+0000 7f3e70ff9640 1 -- 172.21.15.37:0/4034226485 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c004840 con 0x7f3e741f8a10 2024-09-17T04:34:49.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.215+0000 7f3e7a7a8640 1 -- 172.21.15.37:0/4034226485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e740cb9d0 con 0x7f3e741f8a10 2024-09-17T04:34:49.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.215+0000 7f3e7a7a8640 1 -- 172.21.15.37:0/4034226485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e740cbf90 con 0x7f3e741f8a10 2024-09-17T04:34:49.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.216+0000 7f3e7a7a8640 1 -- 172.21.15.37:0/4034226485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3e7406fab0 con 0x7f3e741f8a10 2024-09-17T04:34:49.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.216+0000 7f3e70ff9640 1 -- 172.21.15.37:0/4034226485 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3e6c049430 con 0x7f3e741f8a10 2024-09-17T04:34:49.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.217+0000 7f3e70ff9640 1 --2- 172.21.15.37:0/4034226485 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3e60078bc0 0x7f3e6007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:49.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.217+0000 7f3e70ff9640 1 -- 172.21.15.37:0/4034226485 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5029+0+0 (secure 0 0 0) 0x7f3e6c0bb7d0 con 0x7f3e741f8a10 2024-09-17T04:34:49.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.217+0000 7f3e70ff9640 1 --2- 172.21.15.37:0/4034226485 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x7f3e6007eb70 0x7f3e60080fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:49.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.217+0000 7f3e70ff9640 1 -- 172.21.15.37:0/4034226485 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3e6c02cab0 con 0x7f3e6007eb70 2024-09-17T04:34:49.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.217+0000 7f3e70ff9640 1 -- 172.21.15.37:0/4034226485 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3e6c0c02c0 con 0x7f3e741f8a10 2024-09-17T04:34:49.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.217+0000 7f3e73fff640 1 --2- 172.21.15.37:0/4034226485 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3e60078bc0 0x7f3e6007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:49.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.217+0000 7f3e737fe640 1 --2- 172.21.15.37:0/4034226485 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x7f3e6007eb70 0x7f3e60080fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:49.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.218+0000 7f3e737fe640 1 --2- 172.21.15.37:0/4034226485 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x7f3e6007eb70 0x7f3e60080fc0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:49.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.218+0000 7f3e73fff640 1 --2- 172.21.15.37:0/4034226485 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3e60078bc0 0x7f3e6007b0d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3e64007fd0 tx=0x7f3e64009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:49.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.218+0000 7f3e70ff9640 1 -- 172.21.15.37:0/4034226485 <== osd.2 v2:172.21.15.37:6816/579720176 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3e6c02cab0 con 0x7f3e6007eb70 2024-09-17T04:34:49.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.231+0000 7f3e7a7a8640 1 -- 172.21.15.37:0/4034226485 --> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3e740cb780 con 0x7f3e6007eb70 2024-09-17T04:34:49.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.231+0000 7f3e70ff9640 1 -- 172.21.15.37:0/4034226485 <== osd.2 v2:172.21.15.37:6816/579720176 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3e740cb780 con 0x7f3e6007eb70 2024-09-17T04:34:49.232 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T04:34:49.232 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T04:34:49.232 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T04:34:49.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.232+0000 7f3e7a7a8640 1 -- 172.21.15.37:0/4034226485 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x7f3e6007eb70 msgr2=0x7f3e60080fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:49.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.232+0000 7f3e7a7a8640 1 --2- 172.21.15.37:0/4034226485 >> [v2:172.21.15.37:6816/579720176,v1:172.21.15.37:6817/579720176] conn(0x7f3e6007eb70 0x7f3e60080fc0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.232+0000 7f3e7a7a8640 1 -- 172.21.15.37:0/4034226485 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3e60078bc0 msgr2=0x7f3e6007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.232+0000 7f3e7a7a8640 1 --2- 172.21.15.37:0/4034226485 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3e60078bc0 0x7f3e6007b0d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3e64007fd0 tx=0x7f3e64009040 comp rx=0 tx=0).stop 2024-09-17T04:34:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.232+0000 7f3e7a7a8640 1 -- 172.21.15.37:0/4034226485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e741f8a10 msgr2=0x7f3e741f8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.232+0000 7f3e7a7a8640 1 --2- 172.21.15.37:0/4034226485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e741f8a10 0x7f3e741f8e20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c02b0b0 tx=0x7f3e6c003ce0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.232+0000 7f3e78d45640 1 -- 172.21.15.37:0/4034226485 reap_dead start 2024-09-17T04:34:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.232+0000 7f3e7a7a8640 1 -- 172.21.15.37:0/4034226485 shutdown_connections 2024-09-17T04:34:49.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.232+0000 7f3e7a7a8640 1 -- 172.21.15.37:0/4034226485 >> 172.21.15.37:0/4034226485 conn(0x7f3e7408bef0 msgr2=0x7f3e740b7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:49.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.232+0000 7f3e7a7a8640 1 -- 172.21.15.37:0/4034226485 shutdown_connections 2024-09-17T04:34:49.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.232+0000 7f3e7a7a8640 1 -- 172.21.15.37:0/4034226485 wait complete. 2024-09-17T04:34:49.242 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-17T04:34:49.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.360+0000 7f323c5da640 1 -- 172.21.15.37:0/2037726489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32340d1440 msgr2=0x7f3234162d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:49.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.360+0000 7f323c5da640 1 --2- 172.21.15.37:0/2037726489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32340d1440 0x7f3234162d50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f322c005fd0 tx=0x7f322c02aa90 comp rx=0 tx=0).stop 2024-09-17T04:34:49.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.360+0000 7f323c5da640 1 -- 172.21.15.37:0/2037726489 shutdown_connections 2024-09-17T04:34:49.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.360+0000 7f323c5da640 1 --2- 172.21.15.37:0/2037726489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32340d1440 0x7f3234162d50 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.360+0000 7f323c5da640 1 --2- 172.21.15.37:0/2037726489 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f323406f9f0 0x7f32340d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.360+0000 7f323c5da640 1 -- 172.21.15.37:0/2037726489 >> 172.21.15.37:0/2037726489 conn(0x7f323408bef0 msgr2=0x7f32340d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:49.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.360+0000 7f323c5da640 1 -- 172.21.15.37:0/2037726489 shutdown_connections 2024-09-17T04:34:49.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.360+0000 7f323c5da640 1 -- 172.21.15.37:0/2037726489 wait complete. 2024-09-17T04:34:49.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.362+0000 7f323c5da640 1 Processor -- start 2024-09-17T04:34:49.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.362+0000 7f323c5da640 1 -- start start 2024-09-17T04:34:49.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.363+0000 7f323c5da640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f323406f9f0 0x7f32340a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:49.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.363+0000 7f323c5da640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f32340b3430 0x7f32341f9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:49.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.363+0000 7f323c5da640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32340d1440 0x7f32341f9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:49.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.363+0000 7f323c5da640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f32340b3b00 con 0x7f323406f9f0 2024-09-17T04:34:49.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.363+0000 7f323c5da640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f32340b3980 con 0x7f32340d1440 2024-09-17T04:34:49.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.363+0000 7f323c5da640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f3234163290 con 0x7f32340b3430 2024-09-17T04:34:49.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.363+0000 7f323a376640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f323406f9f0 0x7f32340a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:49.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.363+0000 7f323a376640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f323406f9f0 0x7f32340a1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39748/0 (socket says 172.21.15.37:39748) 2024-09-17T04:34:49.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.363+0000 7f323a376640 1 -- 172.21.15.37:0/2049859663 learned_addr learned my addr 172.21.15.37:0/2049859663 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:49.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.363+0000 7f323ab77640 1 --2- 172.21.15.37:0/2049859663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32340d1440 0x7f32341f9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:49.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.363+0000 7f3239b75640 1 --2- 172.21.15.37:0/2049859663 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f32340b3430 0x7f32341f9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:49.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.363+0000 7f323a376640 1 -- 172.21.15.37:0/2049859663 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f32340b3430 msgr2=0x7f32341f9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:49.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.363+0000 7f323a376640 1 --2- 172.21.15.37:0/2049859663 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f32340b3430 0x7f32341f9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.363+0000 7f323a376640 1 -- 172.21.15.37:0/2049859663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32340d1440 msgr2=0x7f32341f9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:49.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.363+0000 7f323a376640 1 --2- 172.21.15.37:0/2049859663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32340d1440 0x7f32341f9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.363+0000 7f323a376640 1 -- 172.21.15.37:0/2049859663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32340b5010 con 0x7f323406f9f0 2024-09-17T04:34:49.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.363+0000 7f323ab77640 1 --2- 172.21.15.37:0/2049859663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32340d1440 0x7f32341f9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:49.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.364+0000 7f323a376640 1 --2- 172.21.15.37:0/2049859663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f323406f9f0 0x7f32340a1580 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f323000e790 tx=0x7f323000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:49.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.364+0000 7f322affd640 1 -- 172.21.15.37:0/2049859663 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323000eea0 con 0x7f323406f9f0 2024-09-17T04:34:49.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.364+0000 7f322affd640 1 -- 172.21.15.37:0/2049859663 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f323000cdb0 con 0x7f323406f9f0 2024-09-17T04:34:49.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.364+0000 7f323c5da640 1 -- 172.21.15.37:0/2049859663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32340b52f0 con 0x7f323406f9f0 2024-09-17T04:34:49.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.364+0000 7f322affd640 1 -- 172.21.15.37:0/2049859663 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3230010690 con 0x7f323406f9f0 2024-09-17T04:34:49.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.364+0000 7f323c5da640 1 -- 172.21.15.37:0/2049859663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32340b58b0 con 0x7f323406f9f0 2024-09-17T04:34:49.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.365+0000 7f323c5da640 1 -- 172.21.15.37:0/2049859663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32340d34a0 con 0x7f323406f9f0 2024-09-17T04:34:49.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.366+0000 7f322affd640 1 -- 172.21.15.37:0/2049859663 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3230010830 con 0x7f323406f9f0 2024-09-17T04:34:49.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.366+0000 7f322affd640 1 --2- 172.21.15.37:0/2049859663 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3218078bc0 0x7f321807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:49.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.366+0000 7f322affd640 1 -- 172.21.15.37:0/2049859663 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5029+0+0 (secure 0 0 0) 0x7f323009b540 con 0x7f323406f9f0 2024-09-17T04:34:49.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.367+0000 7f3239b75640 1 --2- 172.21.15.37:0/2049859663 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3218078bc0 0x7f321807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:49.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.367+0000 7f3239b75640 1 --2- 172.21.15.37:0/2049859663 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3218078bc0 0x7f321807b0d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3224007fd0 tx=0x7f3224009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:49.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.371+0000 7f322affd640 1 -- 172.21.15.37:0/2049859663 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3230063b80 con 0x7f323406f9f0 2024-09-17T04:34:49.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.556+0000 7f323c5da640 1 -- 172.21.15.37:0/2049859663 --> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f323406a150 con 0x7f3218078bc0 2024-09-17T04:34:49.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.557+0000 7f322affd640 1 -- 172.21.15.37:0/2049859663 <== mgr.4100 v2:172.21.15.92:6800/864337835 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f323406a150 con 0x7f3218078bc0 2024-09-17T04:34:49.557 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 2 to deep-scrub 2024-09-17T04:34:49.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.559+0000 7f323c5da640 1 -- 172.21.15.37:0/2049859663 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3218078bc0 msgr2=0x7f321807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:49.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.559+0000 7f323c5da640 1 --2- 172.21.15.37:0/2049859663 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3218078bc0 0x7f321807b0d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3224007fd0 tx=0x7f3224009040 comp rx=0 tx=0).stop 2024-09-17T04:34:49.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.559+0000 7f323c5da640 1 -- 172.21.15.37:0/2049859663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f323406f9f0 msgr2=0x7f32340a1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:49.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.559+0000 7f323c5da640 1 --2- 172.21.15.37:0/2049859663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f323406f9f0 0x7f32340a1580 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f323000e790 tx=0x7f323000ec50 comp rx=0 tx=0).stop 2024-09-17T04:34:49.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.559+0000 7f323c5da640 1 -- 172.21.15.37:0/2049859663 shutdown_connections 2024-09-17T04:34:49.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.559+0000 7f323c5da640 1 --2- 172.21.15.37:0/2049859663 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3218078bc0 0x7f321807b0d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.559+0000 7f323c5da640 1 --2- 172.21.15.37:0/2049859663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32340d1440 0x7f32341f9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.559+0000 7f323c5da640 1 --2- 172.21.15.37:0/2049859663 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f32340b3430 0x7f32341f9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.559+0000 7f323c5da640 1 --2- 172.21.15.37:0/2049859663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f323406f9f0 0x7f32340a1580 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.559+0000 7f323c5da640 1 -- 172.21.15.37:0/2049859663 >> 172.21.15.37:0/2049859663 conn(0x7f323408bef0 msgr2=0x7f32341625d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:49.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.559+0000 7f323c5da640 1 -- 172.21.15.37:0/2049859663 shutdown_connections 2024-09-17T04:34:49.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.559+0000 7f323c5da640 1 -- 172.21.15.37:0/2049859663 wait complete. 2024-09-17T04:34:49.570 INFO:tasks.ceph:Scrubbing osd.3 2024-09-17T04:34:49.570 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-17T04:34:49.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.692+0000 7fe51bd69640 1 -- 172.21.15.37:0/2297095387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe500003a00 msgr2=0x7fe500003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:49.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.692+0000 7fe51bd69640 1 --2- 172.21.15.37:0/2297095387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe500003a00 0x7fe500003e70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe514072d30 tx=0x7fe500005830 comp rx=0 tx=0).stop 2024-09-17T04:34:49.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.692+0000 7fe51a306640 1 -- 172.21.15.37:0/2297095387 reap_dead start 2024-09-17T04:34:49.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.692+0000 7fe51bd69640 1 -- 172.21.15.37:0/2297095387 shutdown_connections 2024-09-17T04:34:49.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.692+0000 7fe51bd69640 1 -- 172.21.15.37:0/2297095387 >> 172.21.15.37:0/2297095387 conn(0x7fe5140d0a30 msgr2=0x7fe5140d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:49.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.692+0000 7fe51bd69640 1 -- 172.21.15.37:0/2297095387 shutdown_connections 2024-09-17T04:34:49.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.692+0000 7fe51bd69640 1 -- 172.21.15.37:0/2297095387 wait complete. 2024-09-17T04:34:49.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.694+0000 7fe51bd69640 1 Processor -- start 2024-09-17T04:34:49.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.694+0000 7fe51bd69640 1 -- start start 2024-09-17T04:34:49.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.695+0000 7fe51bd69640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe500003a00 0x7fe5141fa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:49.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.695+0000 7fe51bd69640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe51409efa0 0x7fe5140c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:49.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.695+0000 7fe51bd69640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe5140c6320 0x7fe5140c6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.695+0000 7fe51bd69640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe5140cb580 con 0x7fe51409efa0 2024-09-17T04:34:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.695+0000 7fe51bd69640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe5140cb400 con 0x7fe500003a00 2024-09-17T04:34:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.695+0000 7fe51bd69640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fe5140cb700 con 0x7fe5140c6320 2024-09-17T04:34:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.695+0000 7fe519304640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe51409efa0 0x7fe5140c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.695+0000 7fe519304640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe51409efa0 0x7fe5140c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39774/0 (socket says 172.21.15.37:39774) 2024-09-17T04:34:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.695+0000 7fe519304640 1 -- 172.21.15.37:0/3115582852 learned_addr learned my addr 172.21.15.37:0/3115582852 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.695+0000 7fe519b05640 1 --2- 172.21.15.37:0/3115582852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe500003a00 0x7fe5141fa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.695+0000 7fe51a306640 1 --2- 172.21.15.37:0/3115582852 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe5140c6320 0x7fe5140c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.695+0000 7fe519304640 1 -- 172.21.15.37:0/3115582852 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe5140c6320 msgr2=0x7fe5140c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.695+0000 7fe519304640 1 --2- 172.21.15.37:0/3115582852 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe5140c6320 0x7fe5140c6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.695+0000 7fe519304640 1 -- 172.21.15.37:0/3115582852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe500003a00 msgr2=0x7fe5141fa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.695+0000 7fe519304640 1 --2- 172.21.15.37:0/3115582852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe500003a00 0x7fe5141fa0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.695+0000 7fe519304640 1 -- 172.21.15.37:0/3115582852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5140a0ed0 con 0x7fe51409efa0 2024-09-17T04:34:49.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.696+0000 7fe519304640 1 --2- 172.21.15.37:0/3115582852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe51409efa0 0x7fe5140c9760 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe504011ae0 tx=0x7fe504009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:49.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.696+0000 7fe50a7fc640 1 -- 172.21.15.37:0/3115582852 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50401e070 con 0x7fe51409efa0 2024-09-17T04:34:49.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.696+0000 7fe50a7fc640 1 -- 172.21.15.37:0/3115582852 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe50400fe70 con 0x7fe51409efa0 2024-09-17T04:34:49.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.696+0000 7fe51bd69640 1 -- 172.21.15.37:0/3115582852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5140a1170 con 0x7fe51409efa0 2024-09-17T04:34:49.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.696+0000 7fe50a7fc640 1 -- 172.21.15.37:0/3115582852 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe504004290 con 0x7fe51409efa0 2024-09-17T04:34:49.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.696+0000 7fe51bd69640 1 -- 172.21.15.37:0/3115582852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5140a1730 con 0x7fe51409efa0 2024-09-17T04:34:49.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.697+0000 7fe51bd69640 1 -- 172.21.15.37:0/3115582852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe51409f3b0 con 0x7fe51409efa0 2024-09-17T04:34:49.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.698+0000 7fe50a7fc640 1 -- 172.21.15.37:0/3115582852 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe504021070 con 0x7fe51409efa0 2024-09-17T04:34:49.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.698+0000 7fe50a7fc640 1 --2- 172.21.15.37:0/3115582852 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe4f8078b90 0x7fe4f807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:49.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.698+0000 7fe50a7fc640 1 -- 172.21.15.37:0/3115582852 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5029+0+0 (secure 0 0 0) 0x7fe50409c0a0 con 0x7fe51409efa0 2024-09-17T04:34:49.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.698+0000 7fe50a7fc640 1 --2- 172.21.15.37:0/3115582852 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x7fe4f807eb40 0x7fe4f8080f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:49.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.699+0000 7fe50a7fc640 1 -- 172.21.15.37:0/3115582852 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe4f8081660 con 0x7fe4f807eb40 2024-09-17T04:34:49.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.699+0000 7fe50a7fc640 1 -- 172.21.15.37:0/3115582852 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fe50409c530 con 0x7fe51409efa0 2024-09-17T04:34:49.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.699+0000 7fe519b05640 1 --2- 172.21.15.37:0/3115582852 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe4f8078b90 0x7fe4f807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:49.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.699+0000 7fe51a306640 1 --2- 172.21.15.37:0/3115582852 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x7fe4f807eb40 0x7fe4f8080f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:49.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.699+0000 7fe51a306640 1 --2- 172.21.15.37:0/3115582852 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x7fe4f807eb40 0x7fe4f8080f90 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:49.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.699+0000 7fe519b05640 1 --2- 172.21.15.37:0/3115582852 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe4f8078b90 0x7fe4f807b0a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe510033000 tx=0x7fe510038040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:49.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.700+0000 7fe50a7fc640 1 -- 172.21.15.37:0/3115582852 <== osd.3 v2:172.21.15.37:6800/809975826 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4f8081660 con 0x7fe4f807eb40 2024-09-17T04:34:49.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.712+0000 7fe51bd69640 1 -- 172.21.15.37:0/3115582852 --> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe5140ce600 con 0x7fe4f807eb40 2024-09-17T04:34:49.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.713+0000 7fe50a7fc640 1 -- 172.21.15.37:0/3115582852 <== osd.3 v2:172.21.15.37:6800/809975826 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe5140ce600 con 0x7fe4f807eb40 2024-09-17T04:34:49.713 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T04:34:49.713 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T04:34:49.713 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T04:34:49.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.713+0000 7fe51bd69640 1 -- 172.21.15.37:0/3115582852 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x7fe4f807eb40 msgr2=0x7fe4f8080f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:49.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.713+0000 7fe51bd69640 1 --2- 172.21.15.37:0/3115582852 >> [v2:172.21.15.37:6800/809975826,v1:172.21.15.37:6801/809975826] conn(0x7fe4f807eb40 0x7fe4f8080f90 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.713+0000 7fe51bd69640 1 -- 172.21.15.37:0/3115582852 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe4f8078b90 msgr2=0x7fe4f807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:49.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.713+0000 7fe51bd69640 1 --2- 172.21.15.37:0/3115582852 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fe4f8078b90 0x7fe4f807b0a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe510033000 tx=0x7fe510038040 comp rx=0 tx=0).stop 2024-09-17T04:34:49.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.713+0000 7fe51bd69640 1 -- 172.21.15.37:0/3115582852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe51409efa0 msgr2=0x7fe5140c9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:49.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.713+0000 7fe51bd69640 1 --2- 172.21.15.37:0/3115582852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe51409efa0 0x7fe5140c9760 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe504011ae0 tx=0x7fe504009eb0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.713+0000 7fe51a306640 1 -- 172.21.15.37:0/3115582852 reap_dead start 2024-09-17T04:34:49.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.713+0000 7fe51bd69640 1 -- 172.21.15.37:0/3115582852 shutdown_connections 2024-09-17T04:34:49.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.713+0000 7fe51bd69640 1 -- 172.21.15.37:0/3115582852 >> 172.21.15.37:0/3115582852 conn(0x7fe5140d0a30 msgr2=0x7fe5140b4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:49.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.713+0000 7fe51bd69640 1 -- 172.21.15.37:0/3115582852 shutdown_connections 2024-09-17T04:34:49.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.713+0000 7fe51bd69640 1 -- 172.21.15.37:0/3115582852 wait complete. 2024-09-17T04:34:49.723 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-17T04:34:49.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.841+0000 7fcbdf319640 1 -- 172.21.15.37:0/4130660859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbd806f9f0 msgr2=0x7fcbd80d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:49.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.841+0000 7fcbdf319640 1 --2- 172.21.15.37:0/4130660859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbd806f9f0 0x7fcbd80d0f00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fcbd808c9f0 tx=0x7fcbd402a530 comp rx=0 tx=0).stop 2024-09-17T04:34:49.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.841+0000 7fcbdf319640 1 -- 172.21.15.37:0/4130660859 shutdown_connections 2024-09-17T04:34:49.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.841+0000 7fcbdf319640 1 --2- 172.21.15.37:0/4130660859 >> v2:172.21.15.92:3300/0 conn(0x7fcbd80b3430 0x7fcbd80d1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.841+0000 7fcbdf319640 1 --2- 172.21.15.37:0/4130660859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbd806f9f0 0x7fcbd80d0f00 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.841+0000 7fcbdf319640 1 -- 172.21.15.37:0/4130660859 >> 172.21.15.37:0/4130660859 conn(0x7fcbd808bef0 msgr2=0x7fcbd80d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:49.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.841+0000 7fcbdf319640 1 -- 172.21.15.37:0/4130660859 shutdown_connections 2024-09-17T04:34:49.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.841+0000 7fcbdf319640 1 -- 172.21.15.37:0/4130660859 wait complete. 2024-09-17T04:34:49.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.843+0000 7fcbdf319640 1 Processor -- start 2024-09-17T04:34:49.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.843+0000 7fcbdf319640 1 -- start start 2024-09-17T04:34:49.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.844+0000 7fcbdf319640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbd806f9f0 0x7fcbd814da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:49.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.844+0000 7fcbdf319640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbd80b3430 0x7fcbd80cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:49.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.844+0000 7fcbdf319640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcbd8162640 0x7fcbd80c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:49.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.844+0000 7fcbdf319640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd80d4e50 con 0x7fcbd806f9f0 2024-09-17T04:34:49.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.844+0000 7fcbdf319640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd80b3b90 con 0x7fcbd80b3430 2024-09-17T04:34:49.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.844+0000 7fcbdf319640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fcbd80d33c0 con 0x7fcbd8162640 2024-09-17T04:34:49.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.844+0000 7fcbdd0b5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbd806f9f0 0x7fcbd814da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:49.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.844+0000 7fcbdd0b5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbd806f9f0 0x7fcbd814da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39800/0 (socket says 172.21.15.37:39800) 2024-09-17T04:34:49.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.844+0000 7fcbdd0b5640 1 -- 172.21.15.37:0/452654859 learned_addr learned my addr 172.21.15.37:0/452654859 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:49.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.844+0000 7fcbdc8b4640 1 --2- 172.21.15.37:0/452654859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbd80b3430 0x7fcbd80cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:49.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.844+0000 7fcbdd8b6640 1 --2- 172.21.15.37:0/452654859 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcbd8162640 0x7fcbd80c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:49.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.844+0000 7fcbdd0b5640 1 -- 172.21.15.37:0/452654859 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcbd8162640 msgr2=0x7fcbd80c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:49.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.844+0000 7fcbdd0b5640 1 --2- 172.21.15.37:0/452654859 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcbd8162640 0x7fcbd80c85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.844+0000 7fcbdd0b5640 1 -- 172.21.15.37:0/452654859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbd80b3430 msgr2=0x7fcbd80cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:49.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.844+0000 7fcbdd0b5640 1 --2- 172.21.15.37:0/452654859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbd80b3430 0x7fcbd80cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:49.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.844+0000 7fcbdd0b5640 1 -- 172.21.15.37:0/452654859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd814dfa0 con 0x7fcbd806f9f0 2024-09-17T04:34:49.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.845+0000 7fcbdd0b5640 1 --2- 172.21.15.37:0/452654859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbd806f9f0 0x7fcbd814da60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4000c00 tx=0x7fcbd4003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:49.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.845+0000 7fcbcdffb640 1 -- 172.21.15.37:0/452654859 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd403e030 con 0x7fcbd806f9f0 2024-09-17T04:34:49.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.845+0000 7fcbcdffb640 1 -- 172.21.15.37:0/452654859 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcbd4003e00 con 0x7fcbd806f9f0 2024-09-17T04:34:49.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.845+0000 7fcbdf319640 1 -- 172.21.15.37:0/452654859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd80cd780 con 0x7fcbd806f9f0 2024-09-17T04:34:49.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.845+0000 7fcbcdffb640 1 -- 172.21.15.37:0/452654859 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4035040 con 0x7fcbd806f9f0 2024-09-17T04:34:49.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.845+0000 7fcbdf319640 1 -- 172.21.15.37:0/452654859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd80cdf90 con 0x7fcbd806f9f0 2024-09-17T04:34:49.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.846+0000 7fcbdf319640 1 -- 172.21.15.37:0/452654859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbd81632d0 con 0x7fcbd806f9f0 2024-09-17T04:34:49.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.847+0000 7fcbcdffb640 1 -- 172.21.15.37:0/452654859 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcbd4041030 con 0x7fcbd806f9f0 2024-09-17T04:34:49.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.847+0000 7fcbcdffb640 1 --2- 172.21.15.37:0/452654859 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fcbbc078bc0 0x7fcbbc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:49.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.847+0000 7fcbcdffb640 1 -- 172.21.15.37:0/452654859 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5029+0+0 (secure 0 0 0) 0x7fcbd40bb100 con 0x7fcbd806f9f0 2024-09-17T04:34:49.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.848+0000 7fcbdc8b4640 1 --2- 172.21.15.37:0/452654859 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fcbbc078bc0 0x7fcbbc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:49.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.848+0000 7fcbdc8b4640 1 --2- 172.21.15.37:0/452654859 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fcbbc078bc0 0x7fcbbc07b0d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8163130 tx=0x7fcbc800b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:49.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:49.852+0000 7fcbcdffb640 1 -- 172.21.15.37:0/452654859 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fcbd814dfa0 con 0x7fcbd806f9f0 2024-09-17T04:34:50.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.035+0000 7fcbdf319640 1 -- 172.21.15.37:0/452654859 --> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fcbd806a150 con 0x7fcbbc078bc0 2024-09-17T04:34:50.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.035+0000 7fcbcdffb640 1 -- 172.21.15.37:0/452654859 <== mgr.4100 v2:172.21.15.92:6800/864337835 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fcbd806a150 con 0x7fcbbc078bc0 2024-09-17T04:34:50.035 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 3 to deep-scrub 2024-09-17T04:34:50.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.038+0000 7fcbdf319640 1 -- 172.21.15.37:0/452654859 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fcbbc078bc0 msgr2=0x7fcbbc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:50.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.038+0000 7fcbdf319640 1 --2- 172.21.15.37:0/452654859 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fcbbc078bc0 0x7fcbbc07b0d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8163130 tx=0x7fcbc800b2e0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.038+0000 7fcbdf319640 1 -- 172.21.15.37:0/452654859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbd806f9f0 msgr2=0x7fcbd814da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:50.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.038+0000 7fcbdf319640 1 --2- 172.21.15.37:0/452654859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbd806f9f0 0x7fcbd814da60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4000c00 tx=0x7fcbd4003910 comp rx=0 tx=0).stop 2024-09-17T04:34:50.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.038+0000 7fcbdf319640 1 -- 172.21.15.37:0/452654859 shutdown_connections 2024-09-17T04:34:50.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.038+0000 7fcbdf319640 1 --2- 172.21.15.37:0/452654859 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fcbbc078bc0 0x7fcbbc07b0d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.038+0000 7fcbdf319640 1 --2- 172.21.15.37:0/452654859 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcbd8162640 0x7fcbd80c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.038+0000 7fcbdf319640 1 --2- 172.21.15.37:0/452654859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbd80b3430 0x7fcbd80cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.038+0000 7fcbdf319640 1 --2- 172.21.15.37:0/452654859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbd806f9f0 0x7fcbd814da60 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.038+0000 7fcbdf319640 1 -- 172.21.15.37:0/452654859 >> 172.21.15.37:0/452654859 conn(0x7fcbd808bef0 msgr2=0x7fcbd80b7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:50.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.038+0000 7fcbdf319640 1 -- 172.21.15.37:0/452654859 shutdown_connections 2024-09-17T04:34:50.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.038+0000 7fcbdf319640 1 -- 172.21.15.37:0/452654859 wait complete. 2024-09-17T04:34:50.049 INFO:tasks.ceph:Scrubbing osd.4 2024-09-17T04:34:50.049 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-17T04:34:50.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.169+0000 7f1d6b07e640 1 -- 172.21.15.37:0/3542470276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d640cd050 msgr2=0x7f1d640d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:50.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.169+0000 7f1d6b07e640 1 --2- 172.21.15.37:0/3542470276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d640cd050 0x7f1d640d00c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f1d60009ca0 tx=0x7f1d6002a9a0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.169+0000 7f1d6b07e640 1 -- 172.21.15.37:0/3542470276 shutdown_connections 2024-09-17T04:34:50.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.170+0000 7f1d6b07e640 1 --2- 172.21.15.37:0/3542470276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d640cd050 0x7f1d640d00c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.170+0000 7f1d6b07e640 1 -- 172.21.15.37:0/3542470276 >> 172.21.15.37:0/3542470276 conn(0x7f1d640cee70 msgr2=0x7f1d640cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:50.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.170+0000 7f1d6b07e640 1 -- 172.21.15.37:0/3542470276 shutdown_connections 2024-09-17T04:34:50.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.170+0000 7f1d6b07e640 1 -- 172.21.15.37:0/3542470276 wait complete. 2024-09-17T04:34:50.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.171+0000 7f1d6b07e640 1 Processor -- start 2024-09-17T04:34:50.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.171+0000 7f1d6b07e640 1 -- start start 2024-09-17T04:34:50.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.173+0000 7f1d6b07e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d640b7820 0x7f1d6414d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:50.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.173+0000 7f1d6b07e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1d640b7c30 0x7f1d64071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:50.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.173+0000 7f1d6b07e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d640cd050 0x7f1d64072210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:50.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.173+0000 7f1d6b07e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1d640d12a0 con 0x7f1d640b7820 2024-09-17T04:34:50.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.173+0000 7f1d6b07e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1d640d1120 con 0x7f1d640cd050 2024-09-17T04:34:50.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.173+0000 7f1d6b07e640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f1d640d1420 con 0x7f1d640b7c30 2024-09-17T04:34:50.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.173+0000 7f1d68e1a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d640b7820 0x7f1d6414d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:50.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.173+0000 7f1d68e1a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d640b7820 0x7f1d6414d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39830/0 (socket says 172.21.15.37:39830) 2024-09-17T04:34:50.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.173+0000 7f1d68e1a640 1 -- 172.21.15.37:0/1995502902 learned_addr learned my addr 172.21.15.37:0/1995502902 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:50.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.173+0000 7f1d68e1a640 1 -- 172.21.15.37:0/1995502902 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1d640b7c30 msgr2=0x7f1d64071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:50.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.173+0000 7f1d68e1a640 1 --2- 172.21.15.37:0/1995502902 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1d640b7c30 0x7f1d64071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.173+0000 7f1d68e1a640 1 -- 172.21.15.37:0/1995502902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d640cd050 msgr2=0x7f1d64072210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:50.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.173+0000 7f1d6961b640 1 --2- 172.21.15.37:0/1995502902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d640cd050 0x7f1d64072210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:50.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.173+0000 7f1d68e1a640 1 --2- 172.21.15.37:0/1995502902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d640cd050 0x7f1d64072210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.173+0000 7f1d68e1a640 1 -- 172.21.15.37:0/1995502902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d641f84b0 con 0x7f1d640b7820 2024-09-17T04:34:50.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.173+0000 7f1d6961b640 1 --2- 172.21.15.37:0/1995502902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d640cd050 0x7f1d64072210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:50.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.174+0000 7f1d68e1a640 1 --2- 172.21.15.37:0/1995502902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d640b7820 0x7f1d6414d740 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f1d60009dc0 tx=0x7f1d60031f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:50.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.174+0000 7f1d597fa640 1 -- 172.21.15.37:0/1995502902 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d60034040 con 0x7f1d640b7820 2024-09-17T04:34:50.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.174+0000 7f1d597fa640 1 -- 172.21.15.37:0/1995502902 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d60004540 con 0x7f1d640b7820 2024-09-17T04:34:50.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.174+0000 7f1d6b07e640 1 -- 172.21.15.37:0/1995502902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d641f4210 con 0x7f1d640b7820 2024-09-17T04:34:50.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.174+0000 7f1d597fa640 1 -- 172.21.15.37:0/1995502902 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d60045480 con 0x7f1d640b7820 2024-09-17T04:34:50.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.174+0000 7f1d6b07e640 1 -- 172.21.15.37:0/1995502902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d641f47d0 con 0x7f1d640b7820 2024-09-17T04:34:50.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.175+0000 7f1d6b07e640 1 -- 172.21.15.37:0/1995502902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1d6406dcc0 con 0x7f1d640b7820 2024-09-17T04:34:50.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.175+0000 7f1d597fa640 1 -- 172.21.15.37:0/1995502902 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1d60037020 con 0x7f1d640b7820 2024-09-17T04:34:50.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.176+0000 7f1d597fa640 1 --2- 172.21.15.37:0/1995502902 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1d50078bf0 0x7f1d5007b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:50.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.176+0000 7f1d597fa640 1 -- 172.21.15.37:0/1995502902 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5029+0+0 (secure 0 0 0) 0x7f1d600bc810 con 0x7f1d640b7820 2024-09-17T04:34:50.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.176+0000 7f1d597fa640 1 --2- 172.21.15.37:0/1995502902 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x7f1d5007eba0 0x7f1d50080ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:50.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.176+0000 7f1d597fa640 1 -- 172.21.15.37:0/1995502902 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1d60032e90 con 0x7f1d5007eba0 2024-09-17T04:34:50.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.176+0000 7f1d597fa640 1 -- 172.21.15.37:0/1995502902 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f1d600bcca0 con 0x7f1d640b7820 2024-09-17T04:34:50.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.176+0000 7f1d5bfff640 1 --2- 172.21.15.37:0/1995502902 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1d50078bf0 0x7f1d5007b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:50.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.177+0000 7f1d6961b640 1 --2- 172.21.15.37:0/1995502902 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x7f1d5007eba0 0x7f1d50080ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:50.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.177+0000 7f1d5bfff640 1 --2- 172.21.15.37:0/1995502902 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1d50078bf0 0x7f1d5007b100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1d54006fd0 tx=0x7f1d54006bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:50.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.177+0000 7f1d6961b640 1 --2- 172.21.15.37:0/1995502902 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x7f1d5007eba0 0x7f1d50080ff0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:50.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.178+0000 7f1d597fa640 1 -- 172.21.15.37:0/1995502902 <== osd.4 v2:172.21.15.92:6818/2509785752 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1d60032e90 con 0x7f1d5007eba0 2024-09-17T04:34:50.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.190+0000 7f1d6b07e640 1 -- 172.21.15.37:0/1995502902 --> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1d641e83f0 con 0x7f1d5007eba0 2024-09-17T04:34:50.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.191+0000 7f1d597fa640 1 -- 172.21.15.37:0/1995502902 <== osd.4 v2:172.21.15.92:6818/2509785752 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1d641e83f0 con 0x7f1d5007eba0 2024-09-17T04:34:50.191 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T04:34:50.191 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T04:34:50.191 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T04:34:50.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.191+0000 7f1d6b07e640 1 -- 172.21.15.37:0/1995502902 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x7f1d5007eba0 msgr2=0x7f1d50080ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:50.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.191+0000 7f1d6b07e640 1 --2- 172.21.15.37:0/1995502902 >> [v2:172.21.15.92:6818/2509785752,v1:172.21.15.92:6819/2509785752] conn(0x7f1d5007eba0 0x7f1d50080ff0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.191+0000 7f1d6b07e640 1 -- 172.21.15.37:0/1995502902 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1d50078bf0 msgr2=0x7f1d5007b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:50.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.191+0000 7f1d6b07e640 1 --2- 172.21.15.37:0/1995502902 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f1d50078bf0 0x7f1d5007b100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1d54006fd0 tx=0x7f1d54006bc0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.191+0000 7f1d6b07e640 1 -- 172.21.15.37:0/1995502902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d640b7820 msgr2=0x7f1d6414d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:50.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.191+0000 7f1d6b07e640 1 --2- 172.21.15.37:0/1995502902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d640b7820 0x7f1d6414d740 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f1d60009dc0 tx=0x7f1d60031f60 comp rx=0 tx=0).stop 2024-09-17T04:34:50.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.192+0000 7f1d6961b640 1 -- 172.21.15.37:0/1995502902 reap_dead start 2024-09-17T04:34:50.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.192+0000 7f1d6b07e640 1 -- 172.21.15.37:0/1995502902 shutdown_connections 2024-09-17T04:34:50.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.192+0000 7f1d6b07e640 1 -- 172.21.15.37:0/1995502902 >> 172.21.15.37:0/1995502902 conn(0x7f1d640cee70 msgr2=0x7f1d640cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:50.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.192+0000 7f1d6b07e640 1 -- 172.21.15.37:0/1995502902 shutdown_connections 2024-09-17T04:34:50.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.192+0000 7f1d6b07e640 1 -- 172.21.15.37:0/1995502902 wait complete. 2024-09-17T04:34:50.203 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-17T04:34:50.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.320+0000 7f0c02fc2640 1 -- 172.21.15.37:0/1180969458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bfc0d1440 msgr2=0x7f0bfc162d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:50.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.320+0000 7f0c02fc2640 1 --2- 172.21.15.37:0/1180969458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bfc0d1440 0x7f0bfc162d50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4005fd0 tx=0x7f0bf402a8b0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.320+0000 7f0c02fc2640 1 -- 172.21.15.37:0/1180969458 shutdown_connections 2024-09-17T04:34:50.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.320+0000 7f0c02fc2640 1 --2- 172.21.15.37:0/1180969458 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bfc0d1440 0x7f0bfc162d50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.320+0000 7f0c02fc2640 1 --2- 172.21.15.37:0/1180969458 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0bfc06f9f0 0x7f0bfc0d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.320+0000 7f0c02fc2640 1 -- 172.21.15.37:0/1180969458 >> 172.21.15.37:0/1180969458 conn(0x7f0bfc08bef0 msgr2=0x7f0bfc0d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:50.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.320+0000 7f0c02fc2640 1 -- 172.21.15.37:0/1180969458 shutdown_connections 2024-09-17T04:34:50.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.320+0000 7f0c02fc2640 1 -- 172.21.15.37:0/1180969458 wait complete. 2024-09-17T04:34:50.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.322+0000 7f0c02fc2640 1 Processor -- start 2024-09-17T04:34:50.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.322+0000 7f0c02fc2640 1 -- start start 2024-09-17T04:34:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.323+0000 7f0c02fc2640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0bfc06f9f0 0x7f0bfc1f9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.323+0000 7f0c02fc2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bfc0b3430 0x7f0bfc0ce450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.323+0000 7f0c02fc2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bfc0d1440 0x7f0bfc15e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.323+0000 7f0c02fc2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc0b3b00 con 0x7f0bfc0b3430 2024-09-17T04:34:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.323+0000 7f0c02fc2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc0b3980 con 0x7f0bfc0d1440 2024-09-17T04:34:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.323+0000 7f0c02fc2640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f0bfc163290 con 0x7f0bfc06f9f0 2024-09-17T04:34:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.324+0000 7f0bfbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bfc0b3430 0x7f0bfc0ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.324+0000 7f0bfbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bfc0b3430 0x7f0bfc0ce450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39834/0 (socket says 172.21.15.37:39834) 2024-09-17T04:34:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.324+0000 7f0bfbfff640 1 -- 172.21.15.37:0/3323806037 learned_addr learned my addr 172.21.15.37:0/3323806037 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.324+0000 7f0c00d5e640 1 --2- 172.21.15.37:0/3323806037 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0bfc06f9f0 0x7f0bfc1f9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.324+0000 7f0c0155f640 1 --2- 172.21.15.37:0/3323806037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bfc0d1440 0x7f0bfc15e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.324+0000 7f0bfbfff640 1 -- 172.21.15.37:0/3323806037 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0bfc06f9f0 msgr2=0x7f0bfc1f9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.324+0000 7f0bfbfff640 1 --2- 172.21.15.37:0/3323806037 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0bfc06f9f0 0x7f0bfc1f9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.324+0000 7f0bfbfff640 1 -- 172.21.15.37:0/3323806037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bfc0d1440 msgr2=0x7f0bfc15e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.324+0000 7f0bfbfff640 1 --2- 172.21.15.37:0/3323806037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bfc0d1440 0x7f0bfc15e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.324+0000 7f0bfbfff640 1 -- 172.21.15.37:0/3323806037 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bfc1f9c80 con 0x7f0bfc0b3430 2024-09-17T04:34:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.324+0000 7f0bfbfff640 1 --2- 172.21.15.37:0/3323806037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bfc0b3430 0x7f0bfc0ce450 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000c760 tx=0x7f0bf000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.324+0000 7f0bf97fa640 1 -- 172.21.15.37:0/3323806037 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf00050d0 con 0x7f0bfc0b3430 2024-09-17T04:34:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.325+0000 7f0c02fc2640 1 -- 172.21.15.37:0/3323806037 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bfc1f9f60 con 0x7f0bfc0b3430 2024-09-17T04:34:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.325+0000 7f0bf97fa640 1 -- 172.21.15.37:0/3323806037 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0bf0005270 con 0x7f0bfc0b3430 2024-09-17T04:34:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.325+0000 7f0bf97fa640 1 -- 172.21.15.37:0/3323806037 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0003640 con 0x7f0bfc0b3430 2024-09-17T04:34:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.325+0000 7f0c02fc2640 1 -- 172.21.15.37:0/3323806037 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bfc09e330 con 0x7f0bfc0b3430 2024-09-17T04:34:50.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.326+0000 7f0c02fc2640 1 -- 172.21.15.37:0/3323806037 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bfc1f99f0 con 0x7f0bfc0b3430 2024-09-17T04:34:50.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.326+0000 7f0bf97fa640 1 -- 172.21.15.37:0/3323806037 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0bf00037e0 con 0x7f0bfc0b3430 2024-09-17T04:34:50.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.327+0000 7f0bf97fa640 1 --2- 172.21.15.37:0/3323806037 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f0be8078b90 0x7f0be807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:50.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.327+0000 7f0bf97fa640 1 -- 172.21.15.37:0/3323806037 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5029+0+0 (secure 0 0 0) 0x7f0bf009a450 con 0x7f0bfc0b3430 2024-09-17T04:34:50.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.327+0000 7f0c00d5e640 1 --2- 172.21.15.37:0/3323806037 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f0be8078b90 0x7f0be807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:50.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.327+0000 7f0c00d5e640 1 --2- 172.21.15.37:0/3323806037 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f0be8078b90 0x7f0be807b0a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009530 tx=0x7f0bec009260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:50.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.331+0000 7f0bf97fa640 1 -- 172.21.15.37:0/3323806037 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0bfc0d1850 con 0x7f0bfc0b3430 2024-09-17T04:34:50.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.517+0000 7f0c02fc2640 1 -- 172.21.15.37:0/3323806037 --> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f0bfc050f60 con 0x7f0be8078b90 2024-09-17T04:34:50.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.517+0000 7f0bf97fa640 1 -- 172.21.15.37:0/3323806037 <== mgr.4100 v2:172.21.15.92:6800/864337835 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f0bfc050f60 con 0x7f0be8078b90 2024-09-17T04:34:50.517 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 4 to deep-scrub 2024-09-17T04:34:50.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.519+0000 7f0c02fc2640 1 -- 172.21.15.37:0/3323806037 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f0be8078b90 msgr2=0x7f0be807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:50.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.519+0000 7f0c02fc2640 1 --2- 172.21.15.37:0/3323806037 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f0be8078b90 0x7f0be807b0a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009530 tx=0x7f0bec009260 comp rx=0 tx=0).stop 2024-09-17T04:34:50.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.519+0000 7f0c02fc2640 1 -- 172.21.15.37:0/3323806037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bfc0b3430 msgr2=0x7f0bfc0ce450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:50.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.519+0000 7f0c02fc2640 1 --2- 172.21.15.37:0/3323806037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bfc0b3430 0x7f0bfc0ce450 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000c760 tx=0x7f0bf000cc20 comp rx=0 tx=0).stop 2024-09-17T04:34:50.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.519+0000 7f0c02fc2640 1 -- 172.21.15.37:0/3323806037 shutdown_connections 2024-09-17T04:34:50.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.519+0000 7f0c02fc2640 1 --2- 172.21.15.37:0/3323806037 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f0be8078b90 0x7f0be807b0a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.519+0000 7f0c02fc2640 1 --2- 172.21.15.37:0/3323806037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bfc0d1440 0x7f0bfc15e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.519+0000 7f0c02fc2640 1 --2- 172.21.15.37:0/3323806037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bfc0b3430 0x7f0bfc0ce450 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.519+0000 7f0c02fc2640 1 --2- 172.21.15.37:0/3323806037 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0bfc06f9f0 0x7f0bfc1f9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.519+0000 7f0c02fc2640 1 -- 172.21.15.37:0/3323806037 >> 172.21.15.37:0/3323806037 conn(0x7f0bfc08bef0 msgr2=0x7f0bfc0b7480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:50.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.519+0000 7f0c02fc2640 1 -- 172.21.15.37:0/3323806037 shutdown_connections 2024-09-17T04:34:50.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.519+0000 7f0c02fc2640 1 -- 172.21.15.37:0/3323806037 wait complete. 2024-09-17T04:34:50.531 INFO:tasks.ceph:Scrubbing osd.5 2024-09-17T04:34:50.531 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-17T04:34:50.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.652+0000 7f5dc5870640 1 -- 172.21.15.37:0/1406305025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dc00b34c0 msgr2=0x7f5dc0161ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:50.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.652+0000 7f5dc5870640 1 --2- 172.21.15.37:0/1406305025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dc00b34c0 0x7f5dc0161ff0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5db0005fd0 tx=0x7f5db002a8b0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.652+0000 7f5dc5870640 1 -- 172.21.15.37:0/1406305025 shutdown_connections 2024-09-17T04:34:50.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.652+0000 7f5dc5870640 1 --2- 172.21.15.37:0/1406305025 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5dc00b3a10 0x7f5dc00b6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.652+0000 7f5dc5870640 1 --2- 172.21.15.37:0/1406305025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dc00b34c0 0x7f5dc0161ff0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.652+0000 7f5dc5870640 1 -- 172.21.15.37:0/1406305025 >> 172.21.15.37:0/1406305025 conn(0x7f5dc008bef0 msgr2=0x7f5dc00d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.652+0000 7f5dc5870640 1 -- 172.21.15.37:0/1406305025 shutdown_connections 2024-09-17T04:34:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.652+0000 7f5dc5870640 1 -- 172.21.15.37:0/1406305025 wait complete. 2024-09-17T04:34:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.654+0000 7f5dc5870640 1 Processor -- start 2024-09-17T04:34:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.654+0000 7f5dc5870640 1 -- start start 2024-09-17T04:34:50.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.655+0000 7f5dc5870640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dc00b3a10 0x7f5dc01fa390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.655+0000 7f5dc5870640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5dc00b6580 0x7f5dc00b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.655+0000 7f5dc5870640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5dc00a1c80 0x7f5dc00b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.655+0000 7f5dc5870640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc006e770 con 0x7f5dc00b3a10 2024-09-17T04:34:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.655+0000 7f5dc5870640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc006e5f0 con 0x7f5dc00b6580 2024-09-17T04:34:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.655+0000 7f5dc5870640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f5dc006e8f0 con 0x7f5dc00a1c80 2024-09-17T04:34:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.655+0000 7f5dbeffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dc00b3a10 0x7f5dc01fa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.655+0000 7f5dbeffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dc00b3a10 0x7f5dc01fa390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39842/0 (socket says 172.21.15.37:39842) 2024-09-17T04:34:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.655+0000 7f5dbeffd640 1 -- 172.21.15.37:0/4168576616 learned_addr learned my addr 172.21.15.37:0/4168576616 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.655+0000 7f5dbe7fc640 1 --2- 172.21.15.37:0/4168576616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5dc00b6580 0x7f5dc00b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.655+0000 7f5dbf7fe640 1 --2- 172.21.15.37:0/4168576616 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5dc00a1c80 0x7f5dc00b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:50.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.656+0000 7f5dbeffd640 1 -- 172.21.15.37:0/4168576616 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5dc00a1c80 msgr2=0x7f5dc00b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.656+0000 7f5dbeffd640 1 --2- 172.21.15.37:0/4168576616 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5dc00a1c80 0x7f5dc00b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.656+0000 7f5dbeffd640 1 -- 172.21.15.37:0/4168576616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5dc00b6580 msgr2=0x7f5dc00b4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.656+0000 7f5dbeffd640 1 --2- 172.21.15.37:0/4168576616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5dc00b6580 0x7f5dc00b4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.656+0000 7f5dbeffd640 1 -- 172.21.15.37:0/4168576616 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc01fa8d0 con 0x7f5dc00b3a10 2024-09-17T04:34:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.656+0000 7f5dbeffd640 1 --2- 172.21.15.37:0/4168576616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dc00b3a10 0x7f5dc01fa390 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009cd0 tx=0x7f5db00039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.656+0000 7f5da7fff640 1 -- 172.21.15.37:0/4168576616 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0031990 con 0x7f5dc00b3a10 2024-09-17T04:34:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.656+0000 7f5da7fff640 1 -- 172.21.15.37:0/4168576616 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5db0031c20 con 0x7f5dc00b3a10 2024-09-17T04:34:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.656+0000 7f5dc5870640 1 -- 172.21.15.37:0/4168576616 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc01f3d70 con 0x7f5dc00b3a10 2024-09-17T04:34:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.656+0000 7f5da7fff640 1 -- 172.21.15.37:0/4168576616 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0033070 con 0x7f5dc00b3a10 2024-09-17T04:34:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.656+0000 7f5dc5870640 1 -- 172.21.15.37:0/4168576616 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc00a0ec0 con 0x7f5dc00b3a10 2024-09-17T04:34:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.657+0000 7f5dc5870640 1 -- 172.21.15.37:0/4168576616 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5dc006ee10 con 0x7f5dc00b3a10 2024-09-17T04:34:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.658+0000 7f5da7fff640 1 -- 172.21.15.37:0/4168576616 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5db0033210 con 0x7f5dc00b3a10 2024-09-17T04:34:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.658+0000 7f5da7fff640 1 --2- 172.21.15.37:0/4168576616 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f5dac078b80 0x7f5dac07b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.658+0000 7f5da7fff640 1 -- 172.21.15.37:0/4168576616 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5029+0+0 (secure 0 0 0) 0x7f5db00bbd60 con 0x7f5dc00b3a10 2024-09-17T04:34:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.659+0000 7f5da7fff640 1 --2- 172.21.15.37:0/4168576616 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x7f5dac07eb30 0x7f5dac080f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.659+0000 7f5da7fff640 1 -- 172.21.15.37:0/4168576616 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5db000b0b0 con 0x7f5dac07eb30 2024-09-17T04:34:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.659+0000 7f5dbe7fc640 1 --2- 172.21.15.37:0/4168576616 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f5dac078b80 0x7f5dac07b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.659+0000 7f5da7fff640 1 -- 172.21.15.37:0/4168576616 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f5db00be380 con 0x7f5dc00b3a10 2024-09-17T04:34:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.659+0000 7f5dbf7fe640 1 --2- 172.21.15.37:0/4168576616 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x7f5dac07eb30 0x7f5dac080f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.659+0000 7f5dbe7fc640 1 --2- 172.21.15.37:0/4168576616 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f5dac078b80 0x7f5dac07b090 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00b6220 tx=0x7f5db400b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.659+0000 7f5dbf7fe640 1 --2- 172.21.15.37:0/4168576616 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x7f5dac07eb30 0x7f5dac080f80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.660+0000 7f5da7fff640 1 -- 172.21.15.37:0/4168576616 <== osd.5 v2:172.21.15.92:6826/248263832 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5db000b0b0 con 0x7f5dac07eb30 2024-09-17T04:34:50.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.673+0000 7f5dc5870640 1 -- 172.21.15.37:0/4168576616 --> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5dc009ea50 con 0x7f5dac07eb30 2024-09-17T04:34:50.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.674+0000 7f5da7fff640 1 -- 172.21.15.37:0/4168576616 <== osd.5 v2:172.21.15.92:6826/248263832 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5dc009ea50 con 0x7f5dac07eb30 2024-09-17T04:34:50.674 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T04:34:50.674 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T04:34:50.674 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T04:34:50.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.674+0000 7f5dc5870640 1 -- 172.21.15.37:0/4168576616 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x7f5dac07eb30 msgr2=0x7f5dac080f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:50.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.674+0000 7f5dc5870640 1 --2- 172.21.15.37:0/4168576616 >> [v2:172.21.15.92:6826/248263832,v1:172.21.15.92:6827/248263832] conn(0x7f5dac07eb30 0x7f5dac080f80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.674+0000 7f5dc5870640 1 -- 172.21.15.37:0/4168576616 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f5dac078b80 msgr2=0x7f5dac07b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:50.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.674+0000 7f5dc5870640 1 --2- 172.21.15.37:0/4168576616 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f5dac078b80 0x7f5dac07b090 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00b6220 tx=0x7f5db400b2e0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.674+0000 7f5dc5870640 1 -- 172.21.15.37:0/4168576616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dc00b3a10 msgr2=0x7f5dc01fa390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:50.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.674+0000 7f5dc5870640 1 --2- 172.21.15.37:0/4168576616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5dc00b3a10 0x7f5dc01fa390 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009cd0 tx=0x7f5db00039f0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.674+0000 7f5dbf7fe640 1 -- 172.21.15.37:0/4168576616 reap_dead start 2024-09-17T04:34:50.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.674+0000 7f5dc5870640 1 -- 172.21.15.37:0/4168576616 shutdown_connections 2024-09-17T04:34:50.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.674+0000 7f5dc5870640 1 -- 172.21.15.37:0/4168576616 >> 172.21.15.37:0/4168576616 conn(0x7f5dc008bef0 msgr2=0x7f5dc00b2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:50.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.674+0000 7f5dc5870640 1 -- 172.21.15.37:0/4168576616 shutdown_connections 2024-09-17T04:34:50.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.674+0000 7f5dc5870640 1 -- 172.21.15.37:0/4168576616 wait complete. 2024-09-17T04:34:50.685 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-17T04:34:50.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.802+0000 7f3c77fdf640 1 -- 172.21.15.37:0/1152385799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c700b3430 msgr2=0x7f3c700d1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:50.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.802+0000 7f3c77fdf640 1 --2- 172.21.15.37:0/1152385799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c700b3430 0x7f3c700d1440 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3c7008c9f0 tx=0x7f3c6402a5f0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.802+0000 7f3c77fdf640 1 -- 172.21.15.37:0/1152385799 shutdown_connections 2024-09-17T04:34:50.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.802+0000 7f3c77fdf640 1 --2- 172.21.15.37:0/1152385799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c700b3430 0x7f3c700d1440 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.802+0000 7f3c77fdf640 1 --2- 172.21.15.37:0/1152385799 >> v2:172.21.15.92:3300/0 conn(0x7f3c7006f9f0 0x7f3c700d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.802+0000 7f3c77fdf640 1 -- 172.21.15.37:0/1152385799 >> 172.21.15.37:0/1152385799 conn(0x7f3c7008bef0 msgr2=0x7f3c700d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:50.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.802+0000 7f3c77fdf640 1 -- 172.21.15.37:0/1152385799 shutdown_connections 2024-09-17T04:34:50.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.802+0000 7f3c77fdf640 1 -- 172.21.15.37:0/1152385799 wait complete. 2024-09-17T04:34:50.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.804+0000 7f3c77fdf640 1 Processor -- start 2024-09-17T04:34:50.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.804+0000 7f3c77fdf640 1 -- start start 2024-09-17T04:34:50.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.805+0000 7f3c77fdf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c7006f9f0 0x7f3c7015bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:50.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.805+0000 7f3c77fdf640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3c700b3430 0x7f3c7009f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:50.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.805+0000 7f3c77fdf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c70162640 0x7f3c7015e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:50.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.805+0000 7f3c77fdf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3c700b3b00 con 0x7f3c7006f9f0 2024-09-17T04:34:50.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.805+0000 7f3c77fdf640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3c700b3980 con 0x7f3c70162640 2024-09-17T04:34:50.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.805+0000 7f3c77fdf640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f3c700d33c0 con 0x7f3c700b3430 2024-09-17T04:34:50.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.806+0000 7f3c75d7b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c7006f9f0 0x7f3c7015bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:50.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.806+0000 7f3c75d7b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c7006f9f0 0x7f3c7015bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39868/0 (socket says 172.21.15.37:39868) 2024-09-17T04:34:50.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.806+0000 7f3c75d7b640 1 -- 172.21.15.37:0/361527144 learned_addr learned my addr 172.21.15.37:0/361527144 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:50.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.806+0000 7f3c7657c640 1 --2- 172.21.15.37:0/361527144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c70162640 0x7f3c7015e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:50.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.806+0000 7f3c7557a640 1 --2- 172.21.15.37:0/361527144 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3c700b3430 0x7f3c7009f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:50.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.806+0000 7f3c75d7b640 1 -- 172.21.15.37:0/361527144 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3c700b3430 msgr2=0x7f3c7009f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:50.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.806+0000 7f3c75d7b640 1 --2- 172.21.15.37:0/361527144 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3c700b3430 0x7f3c7009f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.806+0000 7f3c75d7b640 1 -- 172.21.15.37:0/361527144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c70162640 msgr2=0x7f3c7015e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:50.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.806+0000 7f3c75d7b640 1 --2- 172.21.15.37:0/361527144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c70162640 0x7f3c7015e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:50.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.806+0000 7f3c75d7b640 1 -- 172.21.15.37:0/361527144 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c700a0050 con 0x7f3c7006f9f0 2024-09-17T04:34:50.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.806+0000 7f3c7657c640 1 --2- 172.21.15.37:0/361527144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c70162640 0x7f3c7015e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:50.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.806+0000 7f3c75d7b640 1 --2- 172.21.15.37:0/361527144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c7006f9f0 0x7f3c7015bb00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3c6000ea50 tx=0x7f3c6000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:50.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.806+0000 7f3c6e7fc640 1 -- 172.21.15.37:0/361527144 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6000cdc0 con 0x7f3c7006f9f0 2024-09-17T04:34:50.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.806+0000 7f3c6e7fc640 1 -- 172.21.15.37:0/361527144 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c60011560 con 0x7f3c7006f9f0 2024-09-17T04:34:50.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.806+0000 7f3c6e7fc640 1 -- 172.21.15.37:0/361527144 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c60011990 con 0x7f3c7006f9f0 2024-09-17T04:34:50.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.806+0000 7f3c77fdf640 1 -- 172.21.15.37:0/361527144 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c7009e3c0 con 0x7f3c7006f9f0 2024-09-17T04:34:50.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.806+0000 7f3c77fdf640 1 -- 172.21.15.37:0/361527144 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c7009e980 con 0x7f3c7006f9f0 2024-09-17T04:34:50.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.807+0000 7f3c77fdf640 1 -- 172.21.15.37:0/361527144 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c70163240 con 0x7f3c7006f9f0 2024-09-17T04:34:50.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.808+0000 7f3c6e7fc640 1 -- 172.21.15.37:0/361527144 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3c60011b30 con 0x7f3c7006f9f0 2024-09-17T04:34:50.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.809+0000 7f3c6e7fc640 1 --2- 172.21.15.37:0/361527144 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3c5c078bc0 0x7f3c5c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:50.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.809+0000 7f3c6e7fc640 1 -- 172.21.15.37:0/361527144 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5029+0+0 (secure 0 0 0) 0x7f3c6009b660 con 0x7f3c7006f9f0 2024-09-17T04:34:50.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.809+0000 7f3c7557a640 1 --2- 172.21.15.37:0/361527144 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3c5c078bc0 0x7f3c5c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:50.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.810+0000 7f3c7557a640 1 --2- 172.21.15.37:0/361527144 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3c5c078bc0 0x7f3c5c07b0d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3c64034000 tx=0x7f3c64004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:50.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.813+0000 7f3c6e7fc640 1 -- 172.21.15.37:0/361527144 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3c60016030 con 0x7f3c7006f9f0 2024-09-17T04:34:50.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.999+0000 7f3c77fdf640 1 -- 172.21.15.37:0/361527144 --> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f3c7006a150 con 0x7f3c5c078bc0 2024-09-17T04:34:50.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:50.999+0000 7f3c6e7fc640 1 -- 172.21.15.37:0/361527144 <== mgr.4100 v2:172.21.15.92:6800/864337835 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3c7006a150 con 0x7f3c5c078bc0 2024-09-17T04:34:50.999 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 5 to deep-scrub 2024-09-17T04:34:51.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.001+0000 7f3c77fdf640 1 -- 172.21.15.37:0/361527144 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3c5c078bc0 msgr2=0x7f3c5c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:51.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.001+0000 7f3c77fdf640 1 --2- 172.21.15.37:0/361527144 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3c5c078bc0 0x7f3c5c07b0d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3c64034000 tx=0x7f3c64004730 comp rx=0 tx=0).stop 2024-09-17T04:34:51.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.001+0000 7f3c77fdf640 1 -- 172.21.15.37:0/361527144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c7006f9f0 msgr2=0x7f3c7015bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:51.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.001+0000 7f3c77fdf640 1 --2- 172.21.15.37:0/361527144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c7006f9f0 0x7f3c7015bb00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3c6000ea50 tx=0x7f3c6000ef10 comp rx=0 tx=0).stop 2024-09-17T04:34:51.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.001+0000 7f3c77fdf640 1 -- 172.21.15.37:0/361527144 shutdown_connections 2024-09-17T04:34:51.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.001+0000 7f3c77fdf640 1 --2- 172.21.15.37:0/361527144 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f3c5c078bc0 0x7f3c5c07b0d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.001+0000 7f3c77fdf640 1 --2- 172.21.15.37:0/361527144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c70162640 0x7f3c7015e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.001+0000 7f3c77fdf640 1 --2- 172.21.15.37:0/361527144 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3c700b3430 0x7f3c7009f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.001+0000 7f3c77fdf640 1 --2- 172.21.15.37:0/361527144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c7006f9f0 0x7f3c7015bb00 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.001+0000 7f3c77fdf640 1 -- 172.21.15.37:0/361527144 >> 172.21.15.37:0/361527144 conn(0x7f3c7008bef0 msgr2=0x7f3c70161be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:51.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.002+0000 7f3c77fdf640 1 -- 172.21.15.37:0/361527144 shutdown_connections 2024-09-17T04:34:51.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.002+0000 7f3c77fdf640 1 -- 172.21.15.37:0/361527144 wait complete. 2024-09-17T04:34:51.013 INFO:tasks.ceph:Scrubbing osd.6 2024-09-17T04:34:51.013 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-17T04:34:51.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.135+0000 7f56d1425640 1 -- 172.21.15.37:0/2525589015 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f56cc0b7820 msgr2=0x7f56cc0d0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:51.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.135+0000 7f56d1425640 1 --2- 172.21.15.37:0/2525589015 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f56cc0b7820 0x7f56cc0d0a10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f56c0005fd0 tx=0x7f56c002a540 comp rx=0 tx=0).stop 2024-09-17T04:34:51.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.135+0000 7f56d1425640 1 -- 172.21.15.37:0/2525589015 shutdown_connections 2024-09-17T04:34:51.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.135+0000 7f56d1425640 1 --2- 172.21.15.37:0/2525589015 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f56cc0b7820 0x7f56cc0d0a10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.135+0000 7f56d1425640 1 --2- 172.21.15.37:0/2525589015 >> v2:172.21.15.92:3300/0 conn(0x7f56cc0cd050 0x7f56cc0d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.135+0000 7f56d1425640 1 -- 172.21.15.37:0/2525589015 >> 172.21.15.37:0/2525589015 conn(0x7f56cc0cee70 msgr2=0x7f56cc0cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:51.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.135+0000 7f56d1425640 1 -- 172.21.15.37:0/2525589015 shutdown_connections 2024-09-17T04:34:51.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.135+0000 7f56d1425640 1 -- 172.21.15.37:0/2525589015 wait complete. 2024-09-17T04:34:51.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.137+0000 7f56d1425640 1 Processor -- start 2024-09-17T04:34:51.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.137+0000 7f56d1425640 1 -- start start 2024-09-17T04:34:51.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.139+0000 7f56d1425640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56cc0b7820 0x7f56cc15c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:51.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.139+0000 7f56d1425640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56cc0b7c30 0x7f56cc0b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:51.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.139+0000 7f56d1425640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f56cc0cd050 0x7f56cc0b2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:51.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.139+0000 7f56d1425640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc160c00 con 0x7f56cc0b7820 2024-09-17T04:34:51.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.139+0000 7f56d1425640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc0cfa70 con 0x7f56cc0b7c30 2024-09-17T04:34:51.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.139+0000 7f56d1425640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f56cc160d80 con 0x7f56cc0cd050 2024-09-17T04:34:51.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.139+0000 7f56caffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56cc0b7820 0x7f56cc15c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:51.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.139+0000 7f56caffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56cc0b7820 0x7f56cc15c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39880/0 (socket says 172.21.15.37:39880) 2024-09-17T04:34:51.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.139+0000 7f56caffd640 1 -- 172.21.15.37:0/2982619250 learned_addr learned my addr 172.21.15.37:0/2982619250 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:51.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.139+0000 7f56ca7fc640 1 --2- 172.21.15.37:0/2982619250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56cc0b7c30 0x7f56cc0b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:51.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.139+0000 7f56cb7fe640 1 --2- 172.21.15.37:0/2982619250 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f56cc0cd050 0x7f56cc0b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:51.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.139+0000 7f56caffd640 1 -- 172.21.15.37:0/2982619250 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f56cc0cd050 msgr2=0x7f56cc0b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:51.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.139+0000 7f56caffd640 1 --2- 172.21.15.37:0/2982619250 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f56cc0cd050 0x7f56cc0b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.139+0000 7f56caffd640 1 -- 172.21.15.37:0/2982619250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56cc0b7c30 msgr2=0x7f56cc0b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:51.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.139+0000 7f56caffd640 1 --2- 172.21.15.37:0/2982619250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56cc0b7c30 0x7f56cc0b22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.139+0000 7f56caffd640 1 -- 172.21.15.37:0/2982619250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56cc0c8e10 con 0x7f56cc0b7820 2024-09-17T04:34:51.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.139+0000 7f56ca7fc640 1 --2- 172.21.15.37:0/2982619250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56cc0b7c30 0x7f56cc0b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:51.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.139+0000 7f56caffd640 1 --2- 172.21.15.37:0/2982619250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56cc0b7820 0x7f56cc15c6e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00e9f0 tx=0x7f56bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:51.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.140+0000 7f56b37fe640 1 -- 172.21.15.37:0/2982619250 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc00cd20 con 0x7f56cc0b7820 2024-09-17T04:34:51.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.140+0000 7f56b37fe640 1 -- 172.21.15.37:0/2982619250 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56bc004020 con 0x7f56cc0b7820 2024-09-17T04:34:51.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.140+0000 7f56d1425640 1 -- 172.21.15.37:0/2982619250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56cc1f3fa0 con 0x7f56cc0b7820 2024-09-17T04:34:51.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.140+0000 7f56b37fe640 1 -- 172.21.15.37:0/2982619250 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc004470 con 0x7f56cc0b7820 2024-09-17T04:34:51.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.140+0000 7f56d1425640 1 -- 172.21.15.37:0/2982619250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56cc1f4560 con 0x7f56cc0b7820 2024-09-17T04:34:51.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.140+0000 7f56d1425640 1 -- 172.21.15.37:0/2982619250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f56cc06dcc0 con 0x7f56cc0b7820 2024-09-17T04:34:51.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.141+0000 7f56b37fe640 1 -- 172.21.15.37:0/2982619250 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f56bc0035f0 con 0x7f56cc0b7820 2024-09-17T04:34:51.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.142+0000 7f56b37fe640 1 --2- 172.21.15.37:0/2982619250 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f56b8078b90 0x7f56b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:51.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.142+0000 7f56b37fe640 1 -- 172.21.15.37:0/2982619250 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5029+0+0 (secure 0 0 0) 0x7f56bc09b3e0 con 0x7f56cc0b7820 2024-09-17T04:34:51.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.142+0000 7f56b37fe640 1 --2- 172.21.15.37:0/2982619250 >> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] conn(0x7f56b807eb40 0x7f56b8080f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:51.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.142+0000 7f56b37fe640 1 -- 172.21.15.37:0/2982619250 --> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f56bc016ec0 con 0x7f56b807eb40 2024-09-17T04:34:51.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.142+0000 7f56b37fe640 1 -- 172.21.15.37:0/2982619250 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f56bc0a2050 con 0x7f56cc0b7820 2024-09-17T04:34:51.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.142+0000 7f56ca7fc640 1 --2- 172.21.15.37:0/2982619250 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f56b8078b90 0x7f56b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:51.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.142+0000 7f56cb7fe640 1 --2- 172.21.15.37:0/2982619250 >> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] conn(0x7f56b807eb40 0x7f56b8080f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:51.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.143+0000 7f56ca7fc640 1 --2- 172.21.15.37:0/2982619250 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f56b8078b90 0x7f56b807b0a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f56c0033000 tx=0x7f56c0004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:51.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.143+0000 7f56cb7fe640 1 --2- 172.21.15.37:0/2982619250 >> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] conn(0x7f56b807eb40 0x7f56b8080f90 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:51.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.144+0000 7f56b37fe640 1 -- 172.21.15.37:0/2982619250 <== osd.6 v2:172.21.15.92:6810/4078365985 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f56bc016ec0 con 0x7f56b807eb40 2024-09-17T04:34:51.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.156+0000 7f56d1425640 1 -- 172.21.15.37:0/2982619250 --> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f56cc0cdcb0 con 0x7f56b807eb40 2024-09-17T04:34:51.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.157+0000 7f56b37fe640 1 -- 172.21.15.37:0/2982619250 <== osd.6 v2:172.21.15.92:6810/4078365985 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f56cc0cdcb0 con 0x7f56b807eb40 2024-09-17T04:34:51.157 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T04:34:51.157 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T04:34:51.157 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T04:34:51.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.157+0000 7f56d1425640 1 -- 172.21.15.37:0/2982619250 >> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] conn(0x7f56b807eb40 msgr2=0x7f56b8080f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:51.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.157+0000 7f56d1425640 1 --2- 172.21.15.37:0/2982619250 >> [v2:172.21.15.92:6810/4078365985,v1:172.21.15.92:6811/4078365985] conn(0x7f56b807eb40 0x7f56b8080f90 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.157+0000 7f56d1425640 1 -- 172.21.15.37:0/2982619250 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f56b8078b90 msgr2=0x7f56b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:51.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.157+0000 7f56d1425640 1 --2- 172.21.15.37:0/2982619250 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f56b8078b90 0x7f56b807b0a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f56c0033000 tx=0x7f56c0004730 comp rx=0 tx=0).stop 2024-09-17T04:34:51.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.157+0000 7f56d1425640 1 -- 172.21.15.37:0/2982619250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56cc0b7820 msgr2=0x7f56cc15c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:51.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.157+0000 7f56d1425640 1 --2- 172.21.15.37:0/2982619250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56cc0b7820 0x7f56cc15c6e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00e9f0 tx=0x7f56bc00eeb0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.157+0000 7f56cb7fe640 1 -- 172.21.15.37:0/2982619250 reap_dead start 2024-09-17T04:34:51.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.157+0000 7f56d1425640 1 -- 172.21.15.37:0/2982619250 shutdown_connections 2024-09-17T04:34:51.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.157+0000 7f56d1425640 1 -- 172.21.15.37:0/2982619250 >> 172.21.15.37:0/2982619250 conn(0x7f56cc0cee70 msgr2=0x7f56cc0cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:51.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.157+0000 7f56d1425640 1 -- 172.21.15.37:0/2982619250 shutdown_connections 2024-09-17T04:34:51.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.157+0000 7f56d1425640 1 -- 172.21.15.37:0/2982619250 wait complete. 2024-09-17T04:34:51.168 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-17T04:34:51.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.285+0000 7fefc2fba640 1 -- 172.21.15.37:0/2815491474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefbc15ced0 msgr2=0x7fefbc15d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:51.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.285+0000 7fefc2fba640 1 --2- 172.21.15.37:0/2815491474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefbc15ced0 0x7fefbc15d2e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fefbc073da0 tx=0x7fefac02a5f0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.285+0000 7fefc2fba640 1 -- 172.21.15.37:0/2815491474 shutdown_connections 2024-09-17T04:34:51.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.285+0000 7fefc2fba640 1 --2- 172.21.15.37:0/2815491474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefbc15ced0 0x7fefbc15d2e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.285+0000 7fefc2fba640 1 --2- 172.21.15.37:0/2815491474 >> v2:172.21.15.92:3300/0 conn(0x7fefbc06d6f0 0x7fefbc15a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.285+0000 7fefc2fba640 1 -- 172.21.15.37:0/2815491474 >> 172.21.15.37:0/2815491474 conn(0x7fefbc0d1330 msgr2=0x7fefbc0d1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:51.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.285+0000 7fefc2fba640 1 -- 172.21.15.37:0/2815491474 shutdown_connections 2024-09-17T04:34:51.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.285+0000 7fefc2fba640 1 -- 172.21.15.37:0/2815491474 wait complete. 2024-09-17T04:34:51.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.287+0000 7fefc2fba640 1 Processor -- start 2024-09-17T04:34:51.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.287+0000 7fefc2fba640 1 -- start start 2024-09-17T04:34:51.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.289+0000 7fefc2fba640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fefbc06d6f0 0x7fefbc1f7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:51.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.289+0000 7fefc2fba640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefbc15ced0 0x7fefbc1f2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:51.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.289+0000 7fefc2fba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefbc160a40 0x7fefbc1f3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:51.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.289+0000 7fefc2fba640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fefbc0cd0b0 con 0x7fefbc160a40 2024-09-17T04:34:51.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.289+0000 7fefc2fba640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fefbc0ccf30 con 0x7fefbc15ced0 2024-09-17T04:34:51.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.289+0000 7fefc2fba640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fefbc0cd230 con 0x7fefbc06d6f0 2024-09-17T04:34:51.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.289+0000 7fefc1557640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefbc160a40 0x7fefbc1f3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:51.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.289+0000 7fefc1557640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefbc160a40 0x7fefbc1f3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39902/0 (socket says 172.21.15.37:39902) 2024-09-17T04:34:51.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.289+0000 7fefc1557640 1 -- 172.21.15.37:0/3850622436 learned_addr learned my addr 172.21.15.37:0/3850622436 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:51.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.289+0000 7fefc0d56640 1 --2- 172.21.15.37:0/3850622436 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fefbc06d6f0 0x7fefbc1f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.289+0000 7fefbbfff640 1 --2- 172.21.15.37:0/3850622436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefbc15ced0 0x7fefbc1f2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.289+0000 7fefc1557640 1 -- 172.21.15.37:0/3850622436 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fefbc06d6f0 msgr2=0x7fefbc1f7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.289+0000 7fefc1557640 1 --2- 172.21.15.37:0/3850622436 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fefbc06d6f0 0x7fefbc1f7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.289+0000 7fefc1557640 1 -- 172.21.15.37:0/3850622436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefbc15ced0 msgr2=0x7fefbc1f2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.289+0000 7fefc1557640 1 --2- 172.21.15.37:0/3850622436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefbc15ced0 0x7fefbc1f2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.289+0000 7fefc1557640 1 -- 172.21.15.37:0/3850622436 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefbc1f7c80 con 0x7fefbc160a40 2024-09-17T04:34:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.289+0000 7fefbbfff640 1 --2- 172.21.15.37:0/3850622436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefbc15ced0 0x7fefbc1f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.289+0000 7fefc0d56640 1 --2- 172.21.15.37:0/3850622436 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fefbc06d6f0 0x7fefbc1f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T04:34:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.289+0000 7fefc1557640 1 --2- 172.21.15.37:0/3850622436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefbc160a40 0x7fefbc1f3350 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fefb400c730 tx=0x7fefb400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.289+0000 7fefb97fa640 1 -- 172.21.15.37:0/3850622436 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4003b90 con 0x7fefbc160a40 2024-09-17T04:34:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.289+0000 7fefb97fa640 1 -- 172.21.15.37:0/3850622436 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefb4003d30 con 0x7fefbc160a40 2024-09-17T04:34:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.289+0000 7fefc2fba640 1 -- 172.21.15.37:0/3850622436 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefbc1f3c00 con 0x7fefbc160a40 2024-09-17T04:34:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.289+0000 7fefb97fa640 1 -- 172.21.15.37:0/3850622436 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4005330 con 0x7fefbc160a40 2024-09-17T04:34:51.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.289+0000 7fefc2fba640 1 -- 172.21.15.37:0/3850622436 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefbc1f41c0 con 0x7fefbc160a40 2024-09-17T04:34:51.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.291+0000 7fefc2fba640 1 -- 172.21.15.37:0/3850622436 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefbc1f3970 con 0x7fefbc160a40 2024-09-17T04:34:51.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.291+0000 7fefb97fa640 1 -- 172.21.15.37:0/3850622436 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fefb40054d0 con 0x7fefbc160a40 2024-09-17T04:34:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.292+0000 7fefb97fa640 1 --2- 172.21.15.37:0/3850622436 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fef9c078b90 0x7fef9c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.292+0000 7fefb97fa640 1 -- 172.21.15.37:0/3850622436 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5029+0+0 (secure 0 0 0) 0x7fefb409a8d0 con 0x7fefbc160a40 2024-09-17T04:34:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.292+0000 7fefc0d56640 1 --2- 172.21.15.37:0/3850622436 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fef9c078b90 0x7fef9c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.292+0000 7fefc0d56640 1 --2- 172.21.15.37:0/3850622436 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fef9c078b90 0x7fef9c07b0a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009aa0 tx=0x7fefa4005db0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:51.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.296+0000 7fefb97fa640 1 -- 172.21.15.37:0/3850622436 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fefbc1f7c80 con 0x7fefbc160a40 2024-09-17T04:34:51.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.482+0000 7fefc2fba640 1 -- 172.21.15.37:0/3850622436 --> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fefbc15f050 con 0x7fef9c078b90 2024-09-17T04:34:51.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.483+0000 7fefb97fa640 1 -- 172.21.15.37:0/3850622436 <== mgr.4100 v2:172.21.15.92:6800/864337835 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fefbc15f050 con 0x7fef9c078b90 2024-09-17T04:34:51.483 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 6 to deep-scrub 2024-09-17T04:34:51.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.485+0000 7fefc2fba640 1 -- 172.21.15.37:0/3850622436 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fef9c078b90 msgr2=0x7fef9c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:51.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.485+0000 7fefc2fba640 1 --2- 172.21.15.37:0/3850622436 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fef9c078b90 0x7fef9c07b0a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009aa0 tx=0x7fefa4005db0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.485+0000 7fefc2fba640 1 -- 172.21.15.37:0/3850622436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefbc160a40 msgr2=0x7fefbc1f3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:51.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.485+0000 7fefc2fba640 1 --2- 172.21.15.37:0/3850622436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefbc160a40 0x7fefbc1f3350 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fefb400c730 tx=0x7fefb400cbf0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.485+0000 7fefc2fba640 1 -- 172.21.15.37:0/3850622436 shutdown_connections 2024-09-17T04:34:51.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.485+0000 7fefc2fba640 1 --2- 172.21.15.37:0/3850622436 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fef9c078b90 0x7fef9c07b0a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.485+0000 7fefc2fba640 1 --2- 172.21.15.37:0/3850622436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefbc160a40 0x7fefbc1f3350 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.485+0000 7fefc2fba640 1 --2- 172.21.15.37:0/3850622436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefbc15ced0 0x7fefbc1f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.485+0000 7fefc2fba640 1 --2- 172.21.15.37:0/3850622436 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fefbc06d6f0 0x7fefbc1f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.485+0000 7fefc2fba640 1 -- 172.21.15.37:0/3850622436 >> 172.21.15.37:0/3850622436 conn(0x7fefbc0d1330 msgr2=0x7fefbc0b6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:51.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.485+0000 7fefc2fba640 1 -- 172.21.15.37:0/3850622436 shutdown_connections 2024-09-17T04:34:51.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.485+0000 7fefc2fba640 1 -- 172.21.15.37:0/3850622436 wait complete. 2024-09-17T04:34:51.496 INFO:tasks.ceph:Scrubbing osd.7 2024-09-17T04:34:51.497 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-17T04:34:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.618+0000 7fb64aeec640 1 -- 172.21.15.37:0/1696593542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb62c0053a0 msgr2=0x7fb62c00ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.618+0000 7fb64aeec640 1 --2- 172.21.15.37:0/1696593542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb62c0053a0 0x7fb62c00ba50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb6440d1f60 tx=0x7fb62c005830 comp rx=0 tx=0).stop 2024-09-17T04:34:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.618+0000 7fb64aeec640 1 -- 172.21.15.37:0/1696593542 shutdown_connections 2024-09-17T04:34:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.618+0000 7fb64aeec640 1 --2- 172.21.15.37:0/1696593542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb62c0053a0 0x7fb62c00ba50 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.618+0000 7fb64aeec640 1 -- 172.21.15.37:0/1696593542 >> 172.21.15.37:0/1696593542 conn(0x7fb6440cee70 msgr2=0x7fb6440cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.618+0000 7fb64aeec640 1 -- 172.21.15.37:0/1696593542 shutdown_connections 2024-09-17T04:34:51.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.618+0000 7fb64aeec640 1 -- 172.21.15.37:0/1696593542 wait complete. 2024-09-17T04:34:51.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.620+0000 7fb64aeec640 1 Processor -- start 2024-09-17T04:34:51.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.620+0000 7fb64aeec640 1 -- start start 2024-09-17T04:34:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.621+0000 7fb64aeec640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb62c0053a0 0x7fb6441e81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.621+0000 7fb64aeec640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb6441f6400 0x7fb6441f6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.621+0000 7fb64aeec640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6441f6810 0x7fb6441f7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.621+0000 7fb64aeec640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb6440d00c0 con 0x7fb6441f6810 2024-09-17T04:34:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.621+0000 7fb64aeec640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb6440cfa50 con 0x7fb62c0053a0 2024-09-17T04:34:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.621+0000 7fb64aeec640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7fb6440d0240 con 0x7fb6441f6400 2024-09-17T04:34:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.621+0000 7fb649489640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6441f6810 0x7fb6441f7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.621+0000 7fb649489640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6441f6810 0x7fb6441f7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39918/0 (socket says 172.21.15.37:39918) 2024-09-17T04:34:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.621+0000 7fb649489640 1 -- 172.21.15.37:0/2547052340 learned_addr learned my addr 172.21.15.37:0/2547052340 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.621+0000 7fb643fff640 1 --2- 172.21.15.37:0/2547052340 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb6441f6400 0x7fb6441f6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.621+0000 7fb648c88640 1 --2- 172.21.15.37:0/2547052340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb62c0053a0 0x7fb6441e81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.622+0000 7fb649489640 1 -- 172.21.15.37:0/2547052340 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb6441f6400 msgr2=0x7fb6441f6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.622+0000 7fb649489640 1 --2- 172.21.15.37:0/2547052340 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb6441f6400 0x7fb6441f6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.622+0000 7fb649489640 1 -- 172.21.15.37:0/2547052340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb62c0053a0 msgr2=0x7fb6441e81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.622+0000 7fb649489640 1 --2- 172.21.15.37:0/2547052340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb62c0053a0 0x7fb6441e81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.622+0000 7fb649489640 1 -- 172.21.15.37:0/2547052340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6440c6d30 con 0x7fb6441f6810 2024-09-17T04:34:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.622+0000 7fb649489640 1 --2- 172.21.15.37:0/2547052340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6441f6810 0x7fb6441f7270 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00c700 tx=0x7fb63c004b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.622+0000 7fb6417fa640 1 -- 172.21.15.37:0/2547052340 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c039070 con 0x7fb6441f6810 2024-09-17T04:34:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.622+0000 7fb6417fa640 1 -- 172.21.15.37:0/2547052340 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb63c02bcc0 con 0x7fb6441f6810 2024-09-17T04:34:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.622+0000 7fb64aeec640 1 -- 172.21.15.37:0/2547052340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6440c7010 con 0x7fb6441f6810 2024-09-17T04:34:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.622+0000 7fb6417fa640 1 -- 172.21.15.37:0/2547052340 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c006290 con 0x7fb6441f6810 2024-09-17T04:34:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.622+0000 7fb64aeec640 1 -- 172.21.15.37:0/2547052340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6440c75d0 con 0x7fb6441f6810 2024-09-17T04:34:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.623+0000 7fb64aeec640 1 -- 172.21.15.37:0/2547052340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb64406dcc0 con 0x7fb6441f6810 2024-09-17T04:34:51.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.624+0000 7fb6417fa640 1 -- 172.21.15.37:0/2547052340 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb63c006430 con 0x7fb6441f6810 2024-09-17T04:34:51.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.624+0000 7fb6417fa640 1 --2- 172.21.15.37:0/2547052340 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb630078b90 0x7fb63007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:51.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.624+0000 7fb6417fa640 1 -- 172.21.15.37:0/2547052340 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5029+0+0 (secure 0 0 0) 0x7fb63c0bb140 con 0x7fb6441f6810 2024-09-17T04:34:51.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.624+0000 7fb648c88640 1 --2- 172.21.15.37:0/2547052340 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb630078b90 0x7fb63007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:51.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.625+0000 7fb6417fa640 1 --2- 172.21.15.37:0/2547052340 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x7fb63007eb40 0x7fb630080f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:51.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.625+0000 7fb6417fa640 1 -- 172.21.15.37:0/2547052340 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb630081660 con 0x7fb63007eb40 2024-09-17T04:34:51.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.625+0000 7fb6417fa640 1 -- 172.21.15.37:0/2547052340 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fb63c0bb5d0 con 0x7fb6441f6810 2024-09-17T04:34:51.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.625+0000 7fb643fff640 1 --2- 172.21.15.37:0/2547052340 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x7fb63007eb40 0x7fb630080f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:51.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.625+0000 7fb648c88640 1 --2- 172.21.15.37:0/2547052340 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb630078b90 0x7fb63007b0a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb634009ca0 tx=0x7fb634005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:51.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.625+0000 7fb643fff640 1 --2- 172.21.15.37:0/2547052340 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x7fb63007eb40 0x7fb630080f90 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:51.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.626+0000 7fb6417fa640 1 -- 172.21.15.37:0/2547052340 <== osd.7 v2:172.21.15.92:6802/3444787223 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb630081660 con 0x7fb63007eb40 2024-09-17T04:34:51.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.639+0000 7fb64aeec640 1 -- 172.21.15.37:0/2547052340 --> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb6440c6ae0 con 0x7fb63007eb40 2024-09-17T04:34:51.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.639+0000 7fb6417fa640 1 -- 172.21.15.37:0/2547052340 <== osd.7 v2:172.21.15.92:6802/3444787223 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb6440c6ae0 con 0x7fb63007eb40 2024-09-17T04:34:51.639 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T04:34:51.640 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T04:34:51.640 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T04:34:51.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.640+0000 7fb64aeec640 1 -- 172.21.15.37:0/2547052340 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x7fb63007eb40 msgr2=0x7fb630080f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:51.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.640+0000 7fb64aeec640 1 --2- 172.21.15.37:0/2547052340 >> [v2:172.21.15.92:6802/3444787223,v1:172.21.15.92:6803/3444787223] conn(0x7fb63007eb40 0x7fb630080f90 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.640+0000 7fb64aeec640 1 -- 172.21.15.37:0/2547052340 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb630078b90 msgr2=0x7fb63007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:51.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.640+0000 7fb64aeec640 1 --2- 172.21.15.37:0/2547052340 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7fb630078b90 0x7fb63007b0a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb634009ca0 tx=0x7fb634005f90 comp rx=0 tx=0).stop 2024-09-17T04:34:51.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.640+0000 7fb64aeec640 1 -- 172.21.15.37:0/2547052340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6441f6810 msgr2=0x7fb6441f7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:51.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.640+0000 7fb64aeec640 1 --2- 172.21.15.37:0/2547052340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6441f6810 0x7fb6441f7270 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00c700 tx=0x7fb63c004b90 comp rx=0 tx=0).stop 2024-09-17T04:34:51.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.640+0000 7fb649489640 1 -- 172.21.15.37:0/2547052340 reap_dead start 2024-09-17T04:34:51.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.640+0000 7fb64aeec640 1 -- 172.21.15.37:0/2547052340 shutdown_connections 2024-09-17T04:34:51.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.640+0000 7fb64aeec640 1 -- 172.21.15.37:0/2547052340 >> 172.21.15.37:0/2547052340 conn(0x7fb6440cee70 msgr2=0x7fb6440b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:51.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.640+0000 7fb64aeec640 1 -- 172.21.15.37:0/2547052340 shutdown_connections 2024-09-17T04:34:51.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.640+0000 7fb64aeec640 1 -- 172.21.15.37:0/2547052340 wait complete. 2024-09-17T04:34:51.650 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-17T04:34:51.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.768+0000 7f9974f7b640 1 -- 172.21.15.37:0/3326779255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99700b74f0 msgr2=0x7f99700b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:51.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.768+0000 7f9974f7b640 1 --2- 172.21.15.37:0/3326779255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99700b74f0 0x7f99700b7900 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f99700cca40 tx=0x7f996402a530 comp rx=0 tx=0).stop 2024-09-17T04:34:51.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.768+0000 7f9974f7b640 1 -- 172.21.15.37:0/3326779255 shutdown_connections 2024-09-17T04:34:51.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.768+0000 7f9974f7b640 1 --2- 172.21.15.37:0/3326779255 >> v2:172.21.15.92:3300/0 conn(0x7f99700cd1b0 0x7f99700cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.768+0000 7f9974f7b640 1 --2- 172.21.15.37:0/3326779255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99700b74f0 0x7f99700b7900 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.768+0000 7f9974f7b640 1 -- 172.21.15.37:0/3326779255 >> 172.21.15.37:0/3326779255 conn(0x7f99700d2460 msgr2=0x7f99700d2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:51.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.768+0000 7f9974f7b640 1 -- 172.21.15.37:0/3326779255 shutdown_connections 2024-09-17T04:34:51.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.768+0000 7f9974f7b640 1 -- 172.21.15.37:0/3326779255 wait complete. 2024-09-17T04:34:51.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.770+0000 7f9974f7b640 1 Processor -- start 2024-09-17T04:34:51.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.770+0000 7f9974f7b640 1 -- start start 2024-09-17T04:34:51.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.771+0000 7f9974f7b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9970072a90 0x7f9970149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:51.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.771+0000 7f9974f7b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99700b74f0 0x7f997014c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:51.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.771+0000 7f9974f7b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f99700cd1b0 0x7f99700c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:51.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.771+0000 7f9974f7b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f99700cdc80 con 0x7f99700b74f0 2024-09-17T04:34:51.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.771+0000 7f9974f7b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f99700cdb00 con 0x7f9970072a90 2024-09-17T04:34:51.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.771+0000 7f9974f7b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f9970161100 con 0x7f99700cd1b0 2024-09-17T04:34:51.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.771+0000 7f996dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99700b74f0 0x7f997014c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:51.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.771+0000 7f996dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99700b74f0 0x7f997014c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39932/0 (socket says 172.21.15.37:39932) 2024-09-17T04:34:51.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.771+0000 7f996dd9b640 1 -- 172.21.15.37:0/2722049552 learned_addr learned my addr 172.21.15.37:0/2722049552 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:51.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.771+0000 7f996e59c640 1 --2- 172.21.15.37:0/2722049552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9970072a90 0x7f9970149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:51.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.771+0000 7f996ed9d640 1 --2- 172.21.15.37:0/2722049552 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f99700cd1b0 0x7f99700c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:51.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.772+0000 7f996dd9b640 1 -- 172.21.15.37:0/2722049552 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f99700cd1b0 msgr2=0x7f99700c5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:51.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.772+0000 7f996dd9b640 1 --2- 172.21.15.37:0/2722049552 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f99700cd1b0 0x7f99700c5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.772+0000 7f996dd9b640 1 -- 172.21.15.37:0/2722049552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9970072a90 msgr2=0x7f9970149160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:51.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.772+0000 7f996dd9b640 1 --2- 172.21.15.37:0/2722049552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9970072a90 0x7f9970149160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.772+0000 7f996dd9b640 1 -- 172.21.15.37:0/2722049552 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99700c7ea0 con 0x7f99700b74f0 2024-09-17T04:34:51.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.772+0000 7f996e59c640 1 --2- 172.21.15.37:0/2722049552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9970072a90 0x7f9970149160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T04:34:51.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.772+0000 7f996dd9b640 1 --2- 172.21.15.37:0/2722049552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99700b74f0 0x7f997014c350 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f99640098f0 tx=0x7f9964033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:51.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.772+0000 7f995effd640 1 -- 172.21.15.37:0/2722049552 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9964005c90 con 0x7f99700b74f0 2024-09-17T04:34:51.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.772+0000 7f995effd640 1 -- 172.21.15.37:0/2722049552 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9964031e50 con 0x7f99700b74f0 2024-09-17T04:34:51.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.772+0000 7f9974f7b640 1 -- 172.21.15.37:0/2722049552 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99700c83a0 con 0x7f99700b74f0 2024-09-17T04:34:51.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.772+0000 7f995effd640 1 -- 172.21.15.37:0/2722049552 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9964005260 con 0x7f99700b74f0 2024-09-17T04:34:51.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.772+0000 7f9974f7b640 1 -- 172.21.15.37:0/2722049552 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99700c8960 con 0x7f99700b74f0 2024-09-17T04:34:51.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.773+0000 7f9974f7b640 1 -- 172.21.15.37:0/2722049552 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99700c8110 con 0x7f99700b74f0 2024-09-17T04:34:51.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.774+0000 7f995effd640 1 -- 172.21.15.37:0/2722049552 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9964005400 con 0x7f99700b74f0 2024-09-17T04:34:51.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.775+0000 7f995effd640 1 --2- 172.21.15.37:0/2722049552 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9950078b90 0x7f995007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:51.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.775+0000 7f995effd640 1 -- 172.21.15.37:0/2722049552 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5029+0+0 (secure 0 0 0) 0x7f99640bbc40 con 0x7f99700b74f0 2024-09-17T04:34:51.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.775+0000 7f996e59c640 1 --2- 172.21.15.37:0/2722049552 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9950078b90 0x7f995007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:51.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.775+0000 7f996e59c640 1 --2- 172.21.15.37:0/2722049552 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9950078b90 0x7f995007b0a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9960007910 tx=0x7f9960009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:51.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.779+0000 7f995effd640 1 -- 172.21.15.37:0/2722049552 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f996403f030 con 0x7f99700b74f0 2024-09-17T04:34:51.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.965+0000 7f9974f7b640 1 -- 172.21.15.37:0/2722049552 --> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f99700b7900 con 0x7f9950078b90 2024-09-17T04:34:51.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.966+0000 7f995effd640 1 -- 172.21.15.37:0/2722049552 <== mgr.4100 v2:172.21.15.92:6800/864337835 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f99700b7900 con 0x7f9950078b90 2024-09-17T04:34:51.965 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 7 to deep-scrub 2024-09-17T04:34:51.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.967+0000 7f9974f7b640 1 -- 172.21.15.37:0/2722049552 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9950078b90 msgr2=0x7f995007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:51.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.967+0000 7f9974f7b640 1 --2- 172.21.15.37:0/2722049552 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9950078b90 0x7f995007b0a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9960007910 tx=0x7f9960009040 comp rx=0 tx=0).stop 2024-09-17T04:34:51.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.967+0000 7f9974f7b640 1 -- 172.21.15.37:0/2722049552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99700b74f0 msgr2=0x7f997014c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:51.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.967+0000 7f9974f7b640 1 --2- 172.21.15.37:0/2722049552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99700b74f0 0x7f997014c350 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f99640098f0 tx=0x7f9964033c20 comp rx=0 tx=0).stop 2024-09-17T04:34:51.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.968+0000 7f9974f7b640 1 -- 172.21.15.37:0/2722049552 shutdown_connections 2024-09-17T04:34:51.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.968+0000 7f9974f7b640 1 --2- 172.21.15.37:0/2722049552 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f9950078b90 0x7f995007b0a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.968+0000 7f9974f7b640 1 --2- 172.21.15.37:0/2722049552 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f99700cd1b0 0x7f99700c5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.968+0000 7f9974f7b640 1 --2- 172.21.15.37:0/2722049552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99700b74f0 0x7f997014c350 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.968+0000 7f9974f7b640 1 --2- 172.21.15.37:0/2722049552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9970072a90 0x7f9970149160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:51.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.968+0000 7f9974f7b640 1 -- 172.21.15.37:0/2722049552 >> 172.21.15.37:0/2722049552 conn(0x7f99700d2460 msgr2=0x7f99700cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:51.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.968+0000 7f9974f7b640 1 -- 172.21.15.37:0/2722049552 shutdown_connections 2024-09-17T04:34:51.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:51.968+0000 7f9974f7b640 1 -- 172.21.15.37:0/2722049552 wait complete. 2024-09-17T04:34:51.979 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:34:52.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.096+0000 7f4495227640 1 -- 172.21.15.37:0/1481144672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44900b3430 msgr2=0x7f44900d28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:52.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.096+0000 7f4495227640 1 --2- 172.21.15.37:0/1481144672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44900b3430 0x7f44900d28f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4488005fd0 tx=0x7f448802aa90 comp rx=0 tx=0).stop 2024-09-17T04:34:52.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.096+0000 7f4495227640 1 -- 172.21.15.37:0/1481144672 shutdown_connections 2024-09-17T04:34:52.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.096+0000 7f4495227640 1 --2- 172.21.15.37:0/1481144672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44900b3430 0x7f44900d28f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:52.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.096+0000 7f4495227640 1 --2- 172.21.15.37:0/1481144672 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f44900d1610 0x7f44900d23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:52.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.096+0000 7f4495227640 1 -- 172.21.15.37:0/1481144672 >> 172.21.15.37:0/1481144672 conn(0x7f449008bef0 msgr2=0x7f44900d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:52.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.096+0000 7f4495227640 1 -- 172.21.15.37:0/1481144672 shutdown_connections 2024-09-17T04:34:52.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.096+0000 7f4495227640 1 -- 172.21.15.37:0/1481144672 wait complete. 2024-09-17T04:34:52.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.098+0000 7f4495227640 1 Processor -- start 2024-09-17T04:34:52.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.098+0000 7f4495227640 1 -- start start 2024-09-17T04:34:52.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.099+0000 7f4495227640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f449006f9f0 0x7f44901625d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:52.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.099+0000 7f4495227640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44900b3430 0x7f4490162b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:52.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.099+0000 7f4495227640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f449014e0b0 0x7f44900cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:52.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.099+0000 7f4495227640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f44900d2e30 con 0x7f44900b3430 2024-09-17T04:34:52.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.099+0000 7f4495227640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f44900b3b90 con 0x7f449006f9f0 2024-09-17T04:34:52.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.099+0000 7f4495227640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f44900d2fb0 con 0x7f449014e0b0 2024-09-17T04:34:52.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.100+0000 7f448e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44900b3430 0x7f4490162b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:52.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.100+0000 7f448e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44900b3430 0x7f4490162b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39938/0 (socket says 172.21.15.37:39938) 2024-09-17T04:34:52.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.100+0000 7f448e59c640 1 -- 172.21.15.37:0/3324226494 learned_addr learned my addr 172.21.15.37:0/3324226494 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:34:52.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.100+0000 7f448f59e640 1 --2- 172.21.15.37:0/3324226494 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f449014e0b0 0x7f44900cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:52.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.100+0000 7f448ed9d640 1 --2- 172.21.15.37:0/3324226494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f449006f9f0 0x7f44901625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:52.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.100+0000 7f448e59c640 1 -- 172.21.15.37:0/3324226494 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f449014e0b0 msgr2=0x7f44900cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:52.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.100+0000 7f448e59c640 1 --2- 172.21.15.37:0/3324226494 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f449014e0b0 0x7f44900cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:52.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.100+0000 7f448e59c640 1 -- 172.21.15.37:0/3324226494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f449006f9f0 msgr2=0x7f44901625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:52.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.100+0000 7f448e59c640 1 --2- 172.21.15.37:0/3324226494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f449006f9f0 0x7f44901625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:52.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.100+0000 7f448e59c640 1 -- 172.21.15.37:0/3324226494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f449015ff30 con 0x7f44900b3430 2024-09-17T04:34:52.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.100+0000 7f448e59c640 1 --2- 172.21.15.37:0/3324226494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44900b3430 0x7f4490162b10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f448400d720 tx=0x7f448400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:52.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.100+0000 7f44777fe640 1 -- 172.21.15.37:0/3324226494 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4484017070 con 0x7f44900b3430 2024-09-17T04:34:52.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.100+0000 7f44777fe640 1 -- 172.21.15.37:0/3324226494 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4484009cc0 con 0x7f44900b3430 2024-09-17T04:34:52.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.100+0000 7f44777fe640 1 -- 172.21.15.37:0/3324226494 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44840106c0 con 0x7f44900b3430 2024-09-17T04:34:52.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.100+0000 7f4495227640 1 -- 172.21.15.37:0/3324226494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44900ccc40 con 0x7f44900b3430 2024-09-17T04:34:52.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.100+0000 7f4495227640 1 -- 172.21.15.37:0/3324226494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44900cd1e0 con 0x7f44900b3430 2024-09-17T04:34:52.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.101+0000 7f4495227640 1 -- 172.21.15.37:0/3324226494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44901514e0 con 0x7f44900b3430 2024-09-17T04:34:52.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.103+0000 7f44777fe640 1 -- 172.21.15.37:0/3324226494 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f448401a070 con 0x7f44900b3430 2024-09-17T04:34:52.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.103+0000 7f44777fe640 1 --2- 172.21.15.37:0/3324226494 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f447c078bc0 0x7f447c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:34:52.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.103+0000 7f44777fe640 1 -- 172.21.15.37:0/3324226494 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5029+0+0 (secure 0 0 0) 0x7f448409abb0 con 0x7f44900b3430 2024-09-17T04:34:52.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.103+0000 7f448ed9d640 1 --2- 172.21.15.37:0/3324226494 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f447c078bc0 0x7f447c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:34:52.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.104+0000 7f448ed9d640 1 --2- 172.21.15.37:0/3324226494 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f447c078bc0 0x7f447c07b0d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4480007fd0 tx=0x7f4480009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:34:52.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.108+0000 7f44777fe640 1 -- 172.21.15.37:0/3324226494 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f449015ff30 con 0x7f44900b3430 2024-09-17T04:34:52.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.288+0000 7f4495227640 1 -- 172.21.15.37:0/3324226494 --> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f44900d34d0 con 0x7f447c078bc0 2024-09-17T04:34:52.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.289+0000 7f44777fe640 1 -- 172.21.15.37:0/3324226494 <== mgr.4100 v2:172.21.15.92:6800/864337835 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13477 (secure 0 0 0) 0x7f44900d34d0 con 0x7f447c078bc0 2024-09-17T04:34:52.289 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:34:52.290 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-17T04:34:52.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.292+0000 7f4495227640 1 -- 172.21.15.37:0/3324226494 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f447c078bc0 msgr2=0x7f447c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:52.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.292+0000 7f4495227640 1 --2- 172.21.15.37:0/3324226494 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f447c078bc0 0x7f447c07b0d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4480007fd0 tx=0x7f4480009040 comp rx=0 tx=0).stop 2024-09-17T04:34:52.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.292+0000 7f4495227640 1 -- 172.21.15.37:0/3324226494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44900b3430 msgr2=0x7f4490162b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:34:52.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.292+0000 7f4495227640 1 --2- 172.21.15.37:0/3324226494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44900b3430 0x7f4490162b10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f448400d720 tx=0x7f448400dbe0 comp rx=0 tx=0).stop 2024-09-17T04:34:52.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.292+0000 7f4495227640 1 -- 172.21.15.37:0/3324226494 shutdown_connections 2024-09-17T04:34:52.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.292+0000 7f4495227640 1 --2- 172.21.15.37:0/3324226494 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f447c078bc0 0x7f447c07b0d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:52.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.292+0000 7f4495227640 1 --2- 172.21.15.37:0/3324226494 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f449014e0b0 0x7f44900cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:52.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.292+0000 7f4495227640 1 --2- 172.21.15.37:0/3324226494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44900b3430 0x7f4490162b10 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:52.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.292+0000 7f4495227640 1 --2- 172.21.15.37:0/3324226494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f449006f9f0 0x7f44901625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:34:52.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.292+0000 7f4495227640 1 -- 172.21.15.37:0/3324226494 >> 172.21.15.37:0/3324226494 conn(0x7f449008bef0 msgr2=0x7f44900b9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:34:52.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.292+0000 7f4495227640 1 -- 172.21.15.37:0/3324226494 shutdown_connections 2024-09-17T04:34:52.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:34:52.292+0000 7f4495227640 1 -- 172.21.15.37:0/3324226494 wait complete. 2024-09-17T04:34:52.303 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-09-17T04:34:50.467760+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":229852,"kb_used_data":5288,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744868,"statfs":{"total":773094113280,"available":772858744832,"internally_reserved":0,"allocated":5414912,"data_stored":2920768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6322167,"num_objects":-15,"num_object_clones":0,"num_object_copies":-30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-15,"num_whiteouts":0,"num_read":-53,"num_read_kb":-39,"num_write":-46,"num_write_kb":-6192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001902"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":154,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T04:34:44.044537+0000","last_change":"2024-09-17T04:33:09.978742+0000","last_active":"2024-09-17T04:34:44.044537+0000","last_peered":"2024-09-17T04:34:44.044537+0000","last_clean":"2024-09-17T04:34:44.044537+0000","last_became_active":"2024-09-17T04:33:09.977496+0000","last_became_peered":"2024-09-17T04:33:09.977496+0000","last_unstale":"2024-09-17T04:34:44.044537+0000","last_undegraded":"2024-09-17T04:34:44.044537+0000","last_fullsized":"2024-09-17T04:34:44.044537+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T04:33:08.930340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T04:33:08.930340+0000","last_clean_scrub_stamp":"2024-09-17T04:33:08.930340+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:21:26.566102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542168,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28128,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343712,"statfs":{"total":96636764160,"available":96607961088,"internally_reserved":0,"allocated":1122304,"data_stored":807872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542168,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":217504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542169,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27548,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":528384,"data_stored":217504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542168,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31708,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":528384,"data_stored":217504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542169,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27548,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":528384,"data_stored":217504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542168,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":217504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542167,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":217504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542168,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1122304,"data_stored":807872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:34:52.306 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-17T04:33:08.930340+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=4, tm_min=33, tm_sec=8, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=4, tm_min=34, tm_sec=47, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T04:34:52.306 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-17T04:35:12.307 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T04:35:12.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.440+0000 7f8b719c6640 1 -- 172.21.15.37:0/3195023868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b6c0cafa0 msgr2=0x7f8b6c0b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:35:12.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.440+0000 7f8b719c6640 1 --2- 172.21.15.37:0/3195023868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b6c0cafa0 0x7f8b6c0b3090 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8b64005fd0 tx=0x7f8b6402aa90 comp rx=0 tx=0).stop 2024-09-17T04:35:12.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.440+0000 7f8b719c6640 1 -- 172.21.15.37:0/3195023868 shutdown_connections 2024-09-17T04:35:12.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.440+0000 7f8b719c6640 1 --2- 172.21.15.37:0/3195023868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b6c0cafa0 0x7f8b6c0b3090 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:35:12.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.440+0000 7f8b719c6640 1 --2- 172.21.15.37:0/3195023868 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8b6c070010 0x7f8b6c0b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:35:12.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.440+0000 7f8b719c6640 1 -- 172.21.15.37:0/3195023868 >> 172.21.15.37:0/3195023868 conn(0x7f8b6c06d3f0 msgr2=0x7f8b6c0d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:35:12.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.440+0000 7f8b719c6640 1 -- 172.21.15.37:0/3195023868 shutdown_connections 2024-09-17T04:35:12.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.440+0000 7f8b719c6640 1 -- 172.21.15.37:0/3195023868 wait complete. 2024-09-17T04:35:12.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.442+0000 7f8b719c6640 1 Processor -- start 2024-09-17T04:35:12.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.442+0000 7f8b719c6640 1 -- start start 2024-09-17T04:35:12.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.444+0000 7f8b719c6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b6c070010 0x7f8b6c15d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:35:12.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.444+0000 7f8b719c6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b6c0af7a0 0x7f8b6c1f2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:35:12.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.444+0000 7f8b719c6640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8b6c0cafa0 0x7f8b6c1f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:35:12.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.444+0000 7f8b719c6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c0b3bb0 con 0x7f8b6c0af7a0 2024-09-17T04:35:12.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.444+0000 7f8b719c6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c0b3a30 con 0x7f8b6c070010 2024-09-17T04:35:12.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.444+0000 7f8b719c6640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f8b6c0b3d30 con 0x7f8b6c0cafa0 2024-09-17T04:35:12.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.444+0000 7f8b6a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b6c0af7a0 0x7f8b6c1f2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:35:12.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.444+0000 7f8b6a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b6c0af7a0 0x7f8b6c1f2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56676/0 (socket says 172.21.15.37:56676) 2024-09-17T04:35:12.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.444+0000 7f8b6a7fc640 1 -- 172.21.15.37:0/335262443 learned_addr learned my addr 172.21.15.37:0/335262443 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:35:12.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.444+0000 7f8b6affd640 1 --2- 172.21.15.37:0/335262443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b6c070010 0x7f8b6c15d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:35:12.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.444+0000 7f8b6b7fe640 1 --2- 172.21.15.37:0/335262443 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8b6c0cafa0 0x7f8b6c1f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:35:12.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.444+0000 7f8b6a7fc640 1 -- 172.21.15.37:0/335262443 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8b6c0cafa0 msgr2=0x7f8b6c1f2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:35:12.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.444+0000 7f8b6a7fc640 1 --2- 172.21.15.37:0/335262443 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8b6c0cafa0 0x7f8b6c1f2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:35:12.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.444+0000 7f8b6a7fc640 1 -- 172.21.15.37:0/335262443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b6c070010 msgr2=0x7f8b6c15d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:35:12.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.444+0000 7f8b6a7fc640 1 --2- 172.21.15.37:0/335262443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b6c070010 0x7f8b6c15d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:35:12.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.444+0000 7f8b6a7fc640 1 -- 172.21.15.37:0/335262443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b6c1591b0 con 0x7f8b6c0af7a0 2024-09-17T04:35:12.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.444+0000 7f8b6a7fc640 1 --2- 172.21.15.37:0/335262443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b6c0af7a0 0x7f8b6c1f2920 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000d700 tx=0x7f8b6000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:35:12.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.445+0000 7f8b53fff640 1 -- 172.21.15.37:0/335262443 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b60017070 con 0x7f8b6c0af7a0 2024-09-17T04:35:12.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.445+0000 7f8b53fff640 1 -- 172.21.15.37:0/335262443 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b60005310 con 0x7f8b6c0af7a0 2024-09-17T04:35:12.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.445+0000 7f8b719c6640 1 -- 172.21.15.37:0/335262443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b6c159490 con 0x7f8b6c0af7a0 2024-09-17T04:35:12.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.445+0000 7f8b53fff640 1 -- 172.21.15.37:0/335262443 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b600097c0 con 0x7f8b6c0af7a0 2024-09-17T04:35:12.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.445+0000 7f8b719c6640 1 -- 172.21.15.37:0/335262443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b6c159a50 con 0x7f8b6c0af7a0 2024-09-17T04:35:12.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.446+0000 7f8b719c6640 1 -- 172.21.15.37:0/335262443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b6c158f40 con 0x7f8b6c0af7a0 2024-09-17T04:35:12.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.447+0000 7f8b53fff640 1 -- 172.21.15.37:0/335262443 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8b60009960 con 0x7f8b6c0af7a0 2024-09-17T04:35:12.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.447+0000 7f8b53fff640 1 --2- 172.21.15.37:0/335262443 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8b58078b90 0x7f8b5807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:35:12.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.447+0000 7f8b53fff640 1 -- 172.21.15.37:0/335262443 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5029+0+0 (secure 0 0 0) 0x7f8b6009ac30 con 0x7f8b6c0af7a0 2024-09-17T04:35:12.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.447+0000 7f8b6affd640 1 --2- 172.21.15.37:0/335262443 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8b58078b90 0x7f8b5807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:35:12.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.448+0000 7f8b6affd640 1 --2- 172.21.15.37:0/335262443 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8b58078b90 0x7f8b5807b0a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c001720 tx=0x7f8b5c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:35:12.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.452+0000 7f8b53fff640 1 -- 172.21.15.37:0/335262443 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8b60062ba0 con 0x7f8b6c0af7a0 2024-09-17T04:35:12.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.632+0000 7f8b719c6640 1 -- 172.21.15.37:0/335262443 --> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8b6c0b1340 con 0x7f8b58078b90 2024-09-17T04:35:12.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.633+0000 7f8b53fff640 1 -- 172.21.15.37:0/335262443 <== mgr.4100 v2:172.21.15.92:6800/864337835 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13344 (secure 0 0 0) 0x7f8b6c0b1340 con 0x7f8b58078b90 2024-09-17T04:35:12.633 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:35:12.634 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-17T04:35:12.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.636+0000 7f8b719c6640 1 -- 172.21.15.37:0/335262443 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8b58078b90 msgr2=0x7f8b5807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:35:12.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.636+0000 7f8b719c6640 1 --2- 172.21.15.37:0/335262443 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8b58078b90 0x7f8b5807b0a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c001720 tx=0x7f8b5c009040 comp rx=0 tx=0).stop 2024-09-17T04:35:12.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.636+0000 7f8b719c6640 1 -- 172.21.15.37:0/335262443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b6c0af7a0 msgr2=0x7f8b6c1f2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:35:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.636+0000 7f8b719c6640 1 --2- 172.21.15.37:0/335262443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b6c0af7a0 0x7f8b6c1f2920 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000d700 tx=0x7f8b6000dbc0 comp rx=0 tx=0).stop 2024-09-17T04:35:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.636+0000 7f8b719c6640 1 -- 172.21.15.37:0/335262443 shutdown_connections 2024-09-17T04:35:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.636+0000 7f8b719c6640 1 --2- 172.21.15.37:0/335262443 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f8b58078b90 0x7f8b5807b0a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:35:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.636+0000 7f8b719c6640 1 --2- 172.21.15.37:0/335262443 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8b6c0cafa0 0x7f8b6c1f2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:35:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.636+0000 7f8b719c6640 1 --2- 172.21.15.37:0/335262443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b6c0af7a0 0x7f8b6c1f2920 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:35:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.636+0000 7f8b719c6640 1 --2- 172.21.15.37:0/335262443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b6c070010 0x7f8b6c15d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:35:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.636+0000 7f8b719c6640 1 -- 172.21.15.37:0/335262443 >> 172.21.15.37:0/335262443 conn(0x7f8b6c06d3f0 msgr2=0x7f8b6c15b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:35:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.636+0000 7f8b719c6640 1 -- 172.21.15.37:0/335262443 shutdown_connections 2024-09-17T04:35:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.636+0000 7f8b719c6640 1 -- 172.21.15.37:0/335262443 wait complete. 2024-09-17T04:35:12.647 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-09-17T04:35:12.472918+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":229836,"kb_used_data":5288,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744884,"statfs":{"total":773094113280,"available":772858761216,"internally_reserved":0,"allocated":5414912,"data_stored":2920768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002834"},"pg_stats":[{"pgid":"1.0","version":"45'74","reported_seq":165,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-17T04:34:52.355287+0000","last_change":"2024-09-17T04:34:52.355162+0000","last_active":"2024-09-17T04:34:52.355287+0000","last_peered":"2024-09-17T04:34:52.355287+0000","last_clean":"2024-09-17T04:34:52.355287+0000","last_became_active":"2024-09-17T04:33:09.977496+0000","last_became_peered":"2024-09-17T04:33:09.977496+0000","last_unstale":"2024-09-17T04:34:52.355287+0000","last_undegraded":"2024-09-17T04:34:52.355287+0000","last_fullsized":"2024-09-17T04:34:52.355287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'74","last_scrub_stamp":"2024-09-17T04:34:52.355039+0000","last_deep_scrub":"45'74","last_deep_scrub_stamp":"2024-09-17T04:34:52.355039+0000","last_clean_scrub_stamp":"2024-09-17T04:34:52.355039+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:02.301059+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542173,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1122304,"data_stored":807872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":217504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542174,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27548,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":528384,"data_stored":217504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31708,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":528384,"data_stored":217504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542174,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27548,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":528384,"data_stored":217504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":217504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542172,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":217504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542173,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1122304,"data_stored":807872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T04:35:12.649 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-17T04:35:12.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.766+0000 7f68147e4640 1 -- 172.21.15.37:0/2007113844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f680c071210 msgr2=0x7f680c0cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:35:12.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.766+0000 7f68147e4640 1 --2- 172.21.15.37:0/2007113844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f680c071210 0x7f680c0cd580 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6808005fd0 tx=0x7f680802a960 comp rx=0 tx=0).stop 2024-09-17T04:35:12.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.766+0000 7f68147e4640 1 -- 172.21.15.37:0/2007113844 shutdown_connections 2024-09-17T04:35:12.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.766+0000 7f68147e4640 1 --2- 172.21.15.37:0/2007113844 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f680c15f2f0 0x7f680c15f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:35:12.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.766+0000 7f68147e4640 1 --2- 172.21.15.37:0/2007113844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f680c071210 0x7f680c0cd580 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:35:12.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.766+0000 7f68147e4640 1 -- 172.21.15.37:0/2007113844 >> 172.21.15.37:0/2007113844 conn(0x7f680c0d2450 msgr2=0x7f680c0d2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:35:12.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.766+0000 7f68147e4640 1 -- 172.21.15.37:0/2007113844 shutdown_connections 2024-09-17T04:35:12.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.766+0000 7f68147e4640 1 -- 172.21.15.37:0/2007113844 wait complete. 2024-09-17T04:35:12.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.768+0000 7f68147e4640 1 Processor -- start 2024-09-17T04:35:12.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.768+0000 7f68147e4640 1 -- start start 2024-09-17T04:35:12.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.769+0000 7f68147e4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f680c071210 0x7f680c15c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:35:12.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.769+0000 7f68147e4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f680c0b7810 0x7f680c0c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:35:12.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.769+0000 7f68147e4640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f680c15f2f0 0x7f680c0c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:35:12.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.769+0000 7f68147e4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f680c15fc40 con 0x7f680c071210 2024-09-17T04:35:12.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.769+0000 7f68147e4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f680c15fdc0 con 0x7f680c0b7810 2024-09-17T04:35:12.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.769+0000 7f68147e4640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f680c15ff40 con 0x7f680c15f2f0 2024-09-17T04:35:12.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.769+0000 7f6812580640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f680c071210 0x7f680c15c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:35:12.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.769+0000 7f6812580640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f680c071210 0x7f680c15c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56690/0 (socket says 172.21.15.37:56690) 2024-09-17T04:35:12.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.769+0000 7f6812580640 1 -- 172.21.15.37:0/3246838947 learned_addr learned my addr 172.21.15.37:0/3246838947 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:35:12.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.770+0000 7f6812d81640 1 --2- 172.21.15.37:0/3246838947 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f680c15f2f0 0x7f680c0c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:35:12.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.770+0000 7f6811d7f640 1 --2- 172.21.15.37:0/3246838947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f680c0b7810 0x7f680c0c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:35:12.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.770+0000 7f6812580640 1 -- 172.21.15.37:0/3246838947 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f680c15f2f0 msgr2=0x7f680c0c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:35:12.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.770+0000 7f6812580640 1 --2- 172.21.15.37:0/3246838947 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f680c15f2f0 0x7f680c0c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:35:12.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.770+0000 7f6812580640 1 -- 172.21.15.37:0/3246838947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f680c0b7810 msgr2=0x7f680c0c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:35:12.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.770+0000 7f6812580640 1 --2- 172.21.15.37:0/3246838947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f680c0b7810 0x7f680c0c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:35:12.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.770+0000 7f6812580640 1 -- 172.21.15.37:0/3246838947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680c0c7850 con 0x7f680c071210 2024-09-17T04:35:12.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.770+0000 7f6812580640 1 --2- 172.21.15.37:0/3246838947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f680c071210 0x7f680c15c280 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f6808009be0 tx=0x7f6808031f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:35:12.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.770+0000 7f6802ffd640 1 -- 172.21.15.37:0/3246838947 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6808034040 con 0x7f680c071210 2024-09-17T04:35:12.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.770+0000 7f6802ffd640 1 -- 172.21.15.37:0/3246838947 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6808004e30 con 0x7f680c071210 2024-09-17T04:35:12.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.770+0000 7f68147e4640 1 -- 172.21.15.37:0/3246838947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f680c0c6a90 con 0x7f680c071210 2024-09-17T04:35:12.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.770+0000 7f6802ffd640 1 -- 172.21.15.37:0/3246838947 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6808004490 con 0x7f680c071210 2024-09-17T04:35:12.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.770+0000 7f68147e4640 1 -- 172.21.15.37:0/3246838947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f680c0c7050 con 0x7f680c071210 2024-09-17T04:35:12.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.771+0000 7f68147e4640 1 -- 172.21.15.37:0/3246838947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f680c15f700 con 0x7f680c071210 2024-09-17T04:35:12.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.772+0000 7f6802ffd640 1 -- 172.21.15.37:0/3246838947 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6808037020 con 0x7f680c071210 2024-09-17T04:35:12.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.773+0000 7f6802ffd640 1 --2- 172.21.15.37:0/3246838947 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f67f0078b90 0x7f67f007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:35:12.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.773+0000 7f6802ffd640 1 -- 172.21.15.37:0/3246838947 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5029+0+0 (secure 0 0 0) 0x7f68080bbde0 con 0x7f680c071210 2024-09-17T04:35:12.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.773+0000 7f6811d7f640 1 --2- 172.21.15.37:0/3246838947 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f67f0078b90 0x7f67f007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:35:12.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.774+0000 7f6811d7f640 1 --2- 172.21.15.37:0/3246838947 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f67f0078b90 0x7f67f007b0a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f67fc006fd0 tx=0x7f67fc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:35:12.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.777+0000 7f6802ffd640 1 -- 172.21.15.37:0/3246838947 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6808084550 con 0x7f680c071210 2024-09-17T04:35:12.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.964+0000 7f68147e4640 1 -- 172.21.15.37:0/3246838947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f680c06a150 con 0x7f680c071210 2024-09-17T04:35:12.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.968+0000 7f6802ffd640 1 -- 172.21.15.37:0/3246838947 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v34)=0 v34) ==== 123+0+0 (secure 0 0 0) 0x7f6808083e50 con 0x7f680c071210 2024-09-17T04:35:12.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.969+0000 7f6802ffd640 1 -- 172.21.15.37:0/3246838947 <== mon.0 v2:172.21.15.37:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f6808083960 con 0x7f680c071210 2024-09-17T04:35:12.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.971+0000 7f68147e4640 1 -- 172.21.15.37:0/3246838947 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f67f0078b90 msgr2=0x7f67f007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:35:12.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.971+0000 7f68147e4640 1 --2- 172.21.15.37:0/3246838947 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f67f0078b90 0x7f67f007b0a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f67fc006fd0 tx=0x7f67fc008040 comp rx=0 tx=0).stop 2024-09-17T04:35:12.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.971+0000 7f68147e4640 1 -- 172.21.15.37:0/3246838947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f680c071210 msgr2=0x7f680c15c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:35:12.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.971+0000 7f68147e4640 1 --2- 172.21.15.37:0/3246838947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f680c071210 0x7f680c15c280 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f6808009be0 tx=0x7f6808031f60 comp rx=0 tx=0).stop 2024-09-17T04:35:12.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.971+0000 7f68147e4640 1 -- 172.21.15.37:0/3246838947 shutdown_connections 2024-09-17T04:35:12.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.971+0000 7f68147e4640 1 --2- 172.21.15.37:0/3246838947 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f67f0078b90 0x7f67f007b0a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:35:12.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.971+0000 7f68147e4640 1 --2- 172.21.15.37:0/3246838947 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f680c15f2f0 0x7f680c0c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:35:12.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.971+0000 7f68147e4640 1 --2- 172.21.15.37:0/3246838947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f680c0b7810 0x7f680c0c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:35:12.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.971+0000 7f68147e4640 1 --2- 172.21.15.37:0/3246838947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f680c071210 0x7f680c15c280 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:35:12.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.971+0000 7f68147e4640 1 -- 172.21.15.37:0/3246838947 >> 172.21.15.37:0/3246838947 conn(0x7f680c0d2450 msgr2=0x7f680c15d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:35:12.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.972+0000 7f68147e4640 1 -- 172.21.15.37:0/3246838947 shutdown_connections 2024-09-17T04:35:12.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:12.972+0000 7f68147e4640 1 -- 172.21.15.37:0/3246838947 wait complete. 2024-09-17T04:35:12.983 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T04:35:12.983 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T04:35:13.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.103+0000 7f417b3c2640 1 -- 172.21.15.37:0/2998645666 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f417406f9f0 msgr2=0x7f41740d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:35:13.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.103+0000 7f417b3c2640 1 --2- 172.21.15.37:0/2998645666 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f417406f9f0 0x7f41740d0f00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4170009860 tx=0x7f417002a5a0 comp rx=0 tx=0).stop 2024-09-17T04:35:13.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.103+0000 7f417b3c2640 1 -- 172.21.15.37:0/2998645666 shutdown_connections 2024-09-17T04:35:13.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.103+0000 7f417b3c2640 1 --2- 172.21.15.37:0/2998645666 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f417406f9f0 0x7f41740d0f00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:35:13.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.103+0000 7f417b3c2640 1 -- 172.21.15.37:0/2998645666 >> 172.21.15.37:0/2998645666 conn(0x7f417408bef0 msgr2=0x7f41740d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:35:13.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.103+0000 7f417b3c2640 1 -- 172.21.15.37:0/2998645666 shutdown_connections 2024-09-17T04:35:13.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.103+0000 7f417b3c2640 1 -- 172.21.15.37:0/2998645666 wait complete. 2024-09-17T04:35:13.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.104+0000 7f417b3c2640 1 Processor -- start 2024-09-17T04:35:13.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.105+0000 7f417b3c2640 1 -- start start 2024-09-17T04:35:13.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.106+0000 7f417b3c2640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f417406f9f0 0x7f41740b6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:35:13.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.106+0000 7f417b3c2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41740b3430 0x7f4174157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:35:13.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.106+0000 7f417b3c2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41740d1440 0x7f4174151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:35:13.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.106+0000 7f417b3c2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f41740b3b00 con 0x7f41740b3430 2024-09-17T04:35:13.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.106+0000 7f417b3c2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f41740b3980 con 0x7f41740d1440 2024-09-17T04:35:13.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.106+0000 7f417b3c2640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 -- 0x7f41740d33c0 con 0x7f417406f9f0 2024-09-17T04:35:13.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.106+0000 7f417895d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41740b3430 0x7f4174157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:35:13.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.106+0000 7f417895d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41740b3430 0x7f4174157a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56702/0 (socket says 172.21.15.37:56702) 2024-09-17T04:35:13.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.106+0000 7f417895d640 1 -- 172.21.15.37:0/1272071993 learned_addr learned my addr 172.21.15.37:0/1272071993 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-17T04:35:13.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.106+0000 7f417995f640 1 --2- 172.21.15.37:0/1272071993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41740d1440 0x7f4174151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:35:13.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.106+0000 7f417915e640 1 --2- 172.21.15.37:0/1272071993 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f417406f9f0 0x7f41740b6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:35:13.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.106+0000 7f417895d640 1 -- 172.21.15.37:0/1272071993 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f417406f9f0 msgr2=0x7f41740b6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:35:13.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.106+0000 7f417895d640 1 --2- 172.21.15.37:0/1272071993 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f417406f9f0 0x7f41740b6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:35:13.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.106+0000 7f417895d640 1 -- 172.21.15.37:0/1272071993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41740d1440 msgr2=0x7f4174151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:35:13.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.106+0000 7f417895d640 1 --2- 172.21.15.37:0/1272071993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41740d1440 0x7f4174151c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:35:13.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.106+0000 7f417895d640 1 -- 172.21.15.37:0/1272071993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41740b5990 con 0x7f41740b3430 2024-09-17T04:35:13.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.107+0000 7f417895d640 1 --2- 172.21.15.37:0/1272071993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41740b3430 0x7f4174157a30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f416400c400 tx=0x7f416400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:35:13.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.107+0000 7f4169ffb640 1 -- 172.21.15.37:0/1272071993 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4164017070 con 0x7f41740b3430 2024-09-17T04:35:13.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.107+0000 7f4169ffb640 1 -- 172.21.15.37:0/1272071993 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f41640052f0 con 0x7f41740b3430 2024-09-17T04:35:13.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.107+0000 7f417b3c2640 1 -- 172.21.15.37:0/1272071993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41740b5c70 con 0x7f41740b3430 2024-09-17T04:35:13.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.107+0000 7f4169ffb640 1 -- 172.21.15.37:0/1272071993 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4164005500 con 0x7f41740b3430 2024-09-17T04:35:13.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.107+0000 7f417b3c2640 1 -- 172.21.15.37:0/1272071993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41740b6230 con 0x7f41740b3430 2024-09-17T04:35:13.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.108+0000 7f417b3c2640 1 -- 172.21.15.37:0/1272071993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41740b5700 con 0x7f41740b3430 2024-09-17T04:35:13.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.109+0000 7f4169ffb640 1 -- 172.21.15.37:0/1272071993 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f416401a070 con 0x7f41740b3430 2024-09-17T04:35:13.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.109+0000 7f4169ffb640 1 --2- 172.21.15.37:0/1272071993 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4158078b90 0x7f415807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T04:35:13.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.109+0000 7f4169ffb640 1 -- 172.21.15.37:0/1272071993 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5029+0+0 (secure 0 0 0) 0x7f4164099e90 con 0x7f41740b3430 2024-09-17T04:35:13.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.110+0000 7f417915e640 1 --2- 172.21.15.37:0/1272071993 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4158078b90 0x7f415807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T04:35:13.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.110+0000 7f417915e640 1 --2- 172.21.15.37:0/1272071993 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4158078b90 0x7f415807b0a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4170034000 tx=0x7f41700046c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T04:35:13.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.114+0000 7f4169ffb640 1 -- 172.21.15.37:0/1272071993 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4164061e80 con 0x7f41740b3430 2024-09-17T04:35:13.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.337+0000 7f417b3c2640 1 -- 172.21.15.37:0/1272071993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41740d3890 con 0x7f41740b3430 2024-09-17T04:35:13.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.338+0000 7f4169ffb640 1 -- 172.21.15.37:0/1272071993 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+3902 (secure 0 0 0) 0x7f41740d3890 con 0x7f41740b3430 2024-09-17T04:35:13.338 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T04:35:13.339 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":31,"btime":"2024-09-17T04:34:42:021085+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4826,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/199192833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":199192833},{"type":"v1","addr":"172.21.15.37:6833","nonce":199192833}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4832,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/4098913371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4098913371},{"type":"v1","addr":"172.21.15.92:6836","nonce":4098913371}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/383658044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":383658044},{"type":"v1","addr":"172.21.15.92:6837","nonce":383658044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/399243318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":399243318},{"type":"v1","addr":"172.21.15.37:6835","nonce":399243318}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[]} 2024-09-17T04:35:13.339 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 31 2024-09-17T04:35:13.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.340+0000 7f417b3c2640 1 -- 172.21.15.37:0/1272071993 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4158078b90 msgr2=0x7f415807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:35:13.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.340+0000 7f417b3c2640 1 --2- 172.21.15.37:0/1272071993 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4158078b90 0x7f415807b0a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4170034000 tx=0x7f41700046c0 comp rx=0 tx=0).stop 2024-09-17T04:35:13.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.340+0000 7f417b3c2640 1 -- 172.21.15.37:0/1272071993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41740b3430 msgr2=0x7f4174157a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T04:35:13.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.340+0000 7f417b3c2640 1 --2- 172.21.15.37:0/1272071993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41740b3430 0x7f4174157a30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f416400c400 tx=0x7f416400c8c0 comp rx=0 tx=0).stop 2024-09-17T04:35:13.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.340+0000 7f417b3c2640 1 -- 172.21.15.37:0/1272071993 shutdown_connections 2024-09-17T04:35:13.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.340+0000 7f417b3c2640 1 --2- 172.21.15.37:0/1272071993 >> [v2:172.21.15.92:6800/864337835,v1:172.21.15.92:6801/864337835] conn(0x7f4158078b90 0x7f415807b0a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:35:13.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.340+0000 7f417b3c2640 1 --2- 172.21.15.37:0/1272071993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41740d1440 0x7f4174151c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:35:13.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.340+0000 7f417b3c2640 1 --2- 172.21.15.37:0/1272071993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41740b3430 0x7f4174157a30 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:35:13.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.340+0000 7f417b3c2640 1 --2- 172.21.15.37:0/1272071993 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f417406f9f0 0x7f41740b6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T04:35:13.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.340+0000 7f417b3c2640 1 -- 172.21.15.37:0/1272071993 >> 172.21.15.37:0/1272071993 conn(0x7f417408bef0 msgr2=0x7f41740b2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T04:35:13.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.340+0000 7f417b3c2640 1 -- 172.21.15.37:0/1272071993 shutdown_connections 2024-09-17T04:35:13.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T04:35:13.340+0000 7f417b3c2640 1 -- 172.21.15.37:0/1272071993 wait complete. 2024-09-17T04:35:13.351 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-17T04:35:13.351 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-17T04:35:13.352 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T04:35:13.352 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:35:13.364 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T04:35:13.365 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T04:35:13.365 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:35:13.414 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T04:35:13.415 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T04:35:13.415 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:35:13.426 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T04:35:13.426 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T04:35:13.427 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:35:13.478 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T04:35:13.479 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-17T04:35:13.479 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-17T04:35:13.479 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:35:13.509 INFO:tasks.ceph.osd.0:Stopped 2024-09-17T04:35:13.509 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-17T04:35:13.509 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:35:13.582 INFO:tasks.ceph.osd.1:Stopped 2024-09-17T04:35:13.582 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-17T04:35:13.582 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:35:13.651 INFO:tasks.ceph.osd.2:Stopped 2024-09-17T04:35:13.651 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-17T04:35:13.651 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:35:13.725 INFO:tasks.ceph.osd.3:Stopped 2024-09-17T04:35:13.725 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-17T04:35:13.725 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:35:13.759 INFO:tasks.ceph.osd.4:Stopped 2024-09-17T04:35:13.759 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-17T04:35:13.760 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:35:13.836 INFO:tasks.ceph.osd.5:Stopped 2024-09-17T04:35:13.837 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-17T04:35:13.837 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:35:13.905 INFO:tasks.ceph.osd.6:Stopped 2024-09-17T04:35:13.905 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-17T04:35:13.906 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:35:13.980 INFO:tasks.ceph.osd.7:Stopped 2024-09-17T04:35:13.980 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-17T04:35:13.981 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-17T04:35:13.981 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:35:14.008 INFO:tasks.ceph.mgr.y:Stopped 2024-09-17T04:35:14.009 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-17T04:35:14.009 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:35:14.050 INFO:tasks.ceph.mgr.x:Stopped 2024-09-17T04:35:14.050 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-17T04:35:14.051 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-17T04:35:14.051 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:35:14.063 INFO:tasks.ceph.mon.a:Stopped 2024-09-17T04:35:14.063 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-17T04:35:14.063 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:35:14.103 INFO:tasks.ceph.mon.b:Stopped 2024-09-17T04:35:14.104 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-17T04:35:14.104 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T04:35:14.156 INFO:tasks.ceph.mon.c:Stopped 2024-09-17T04:35:14.156 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-17T04:35:14.157 DEBUG:teuthology.orchestra.run.smithi037:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'bad backtrace' | egrep -v 'object missing on disk' | egrep -v 'error reading table object' | egrep -v 'error reading sessionmap' | egrep -v 'unmatched fragstat' | egrep -v 'unmatched rstat' | egrep -v 'was unreadable, recreating it now' | egrep -v 'Scrub error on inode' | egrep -v 'Metadata damage detected' | egrep -v MDS_FAILED | egrep -v MDS_DAMAGE | head -n 1 2024-09-17T04:35:14.199 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi037.front.sepia.ceph.com 2024-09-17T04:35:14.199 DEBUG:teuthology.orchestra.run.smithi037:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-17T04:35:14.399 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi037.front.sepia.ceph.com 2024-09-17T04:35:14.399 DEBUG:teuthology.orchestra.run.smithi037:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-17T04:35:14.527 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi037.front.sepia.ceph.com 2024-09-17T04:35:14.527 DEBUG:teuthology.orchestra.run.smithi037:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-17T04:35:14.584 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi037.front.sepia.ceph.com 2024-09-17T04:35:14.584 DEBUG:teuthology.orchestra.run.smithi037:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-17T04:35:14.635 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-17T04:35:14.635 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-17T04:35:14.636 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:35:14.638 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:35:14.797 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-17T04:35:14.797 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-17T04:35:14.897 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-17T04:35:14.897 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-17T04:35:14.962 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-17T04:35:14.962 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-17T04:35:15.037 INFO:tasks.ceph:Archiving mon data... 2024-09-17T04:35:15.038 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908540/data/mon.a.tgz 2024-09-17T04:35:15.038 DEBUG:teuthology.orchestra.run.smithi037:> mktemp 2024-09-17T04:35:15.055 INFO:teuthology.orchestra.run.smithi037.stdout:/tmp/tmp.SHO9m3bPsx 2024-09-17T04:35:15.055 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.SHO9m3bPsx 2024-09-17T04:35:15.200 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0666 /tmp/tmp.SHO9m3bPsx 2024-09-17T04:35:15.285 DEBUG:teuthology.orchestra.remote:smithi037:/tmp/tmp.SHO9m3bPsx is 319KB 2024-09-17T04:35:15.381 DEBUG:teuthology.orchestra.run.smithi037:> rm -fr /tmp/tmp.SHO9m3bPsx 2024-09-17T04:35:15.397 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908540/data/mon.b.tgz 2024-09-17T04:35:15.397 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2024-09-17T04:35:15.416 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.7BFmqPVPUg 2024-09-17T04:35:15.417 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.7BFmqPVPUg 2024-09-17T04:35:15.578 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.7BFmqPVPUg 2024-09-17T04:35:15.662 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.7BFmqPVPUg is 428KB 2024-09-17T04:35:15.760 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.7BFmqPVPUg 2024-09-17T04:35:15.775 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908540/data/mon.c.tgz 2024-09-17T04:35:15.775 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2024-09-17T04:35:15.830 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.hcFUfVMyiM 2024-09-17T04:35:15.831 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.hcFUfVMyiM 2024-09-17T04:35:15.974 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.hcFUfVMyiM 2024-09-17T04:35:16.057 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.hcFUfVMyiM is 428KB 2024-09-17T04:35:16.126 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.hcFUfVMyiM 2024-09-17T04:35:16.141 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-17T04:35:16.142 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T04:35:16.144 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T04:35:16.230 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T04:35:16.260 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T04:35:16.260 INFO:tasks.ceph:Archiving crash dumps... 2024-09-17T04:35:16.262 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908540/remote/smithi037/crash 2024-09-17T04:35:16.263 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T04:35:16.299 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908540/remote/smithi092/crash 2024-09-17T04:35:16.300 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T04:35:16.324 INFO:tasks.ceph:Shutting down logrotate 2024-09-17T04:35:16.325 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:35:16.344 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T04:35:16.416 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-17T04:35:16.444 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-17T04:35:16.469 INFO:tasks.ceph:Compressing logs... 2024-09-17T04:35:16.469 DEBUG:teuthology.orchestra.run.smithi037:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T04:35:16.487 DEBUG:teuthology.orchestra.run.smithi092:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T04:35:16.511 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-17T04:35:16.512 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43630.log 2024-09-17T04:35:16.514 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51532.log/var/log/ceph/ceph-osd.0.log: 2024-09-17T04:35:16.514 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47735.log 2024-09-17T04:35:16.514 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51532.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43630.log.gz 2024-09-17T04:35:16.514 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43328.log 2024-09-17T04:35:16.514 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51532.log.gz 2024-09-17T04:35:16.514 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51187.log 2024-09-17T04:35:16.514 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47735.log.gz 2024-09-17T04:35:16.515 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5/var/log/ceph/ceph-client.admin.43328.log: --verbose -- /var/log/ceph/ceph-client.0.48708.log 2024-09-17T04:35:16.515 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36824.log 2024-09-17T04:35:16.515 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43328.log.gz 2024-09-17T04:35:16.515 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51187.log.gz 2024-09-17T04:35:16.515 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.48708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46638.log 2024-09-17T04:35:16.554 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42417.log 2024-09-17T04:35:16.554 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-17T04:35:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.1.42417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36893.log 2024-09-17T04:35:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36732.log 2024-09-17T04:35:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.36893.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-17T04:35:16.555 INFO:teuthology.orchestra.run.smithi092.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.1.42417.log.gz 2024-09-17T04:35:16.555 INFO:teuthology.orchestra.run.smithi092.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36893.log.gz 2024-09-17T04:35:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.36732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36853.log 2024-09-17T04:35:16.555 INFO:teuthology.orchestra.run.smithi092.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36732.log.gz 2024-09-17T04:35:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-17T04:35:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.36853.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T04:35:16.555 INFO:teuthology.orchestra.run.smithi092.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36853.log.gz 2024-09-17T04:35:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-17T04:35:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36773.log 2024-09-17T04:35:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36973.log 2024-09-17T04:35:16.556 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.36773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36933.log 2024-09-17T04:35:16.556 INFO:teuthology.orchestra.run.smithi092.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36773.log.gz/var/log/ceph/ceph-client.admin.36973.log: 2024-09-17T04:35:16.556 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36973.log.gz 2024-09-17T04:35:16.556 INFO:teuthology.orchestra.run.smithi092.stderr: --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-17T04:35:16.556 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.36933.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36933.log.gz 2024-09-17T04:35:16.556 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-09-17T04:35:16.556 INFO:teuthology.orchestra.run.smithi092.stderr: 93.6%/var/log/ceph/ceph-osd.6.log: -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T04:35:16.556 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T04:35:16.556 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42872.log 2024-09-17T04:35:16.556 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-17T04:35:16.556 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.3.42872.log: 87.7% 91.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T04:35:16.556 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.3.42872.log.gz 2024-09-17T04:35:16.556 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-17T04:35:16.557 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36692.log 2024-09-17T04:35:16.557 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-mgr.x.log: 92.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-17T04:35:16.557 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-17T04:35:16.557 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.36692.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36692.log.gz 2024-09-17T04:35:16.557 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36813.log 2024-09-17T04:35:16.557 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.36813.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36813.log.gz 2024-09-17T04:35:16.559 INFO:teuthology.orchestra.run.smithi092.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-17T04:35:16.577 INFO:teuthology.orchestra.run.smithi092.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-17T04:35:16.616 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49965.log 2024-09-17T04:35:16.616 INFO:teuthology.orchestra.run.smithi037.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36824.log.gz 2024-09-17T04:35:16.616 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50425.log 2024-09-17T04:35:16.617 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46638.log.gz 2024-09-17T04:35:16.617 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36820.log 2024-09-17T04:35:16.617 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49965.log: /var/log/ceph/ceph-client.admin.50425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45985.log 2024-09-17T04:35:16.617 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36820.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.49965.log.gz 2024-09-17T04:35:16.617 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50425.log.gz 2024-09-17T04:35:16.617 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36822.log 2024-09-17T04:35:16.617 INFO:teuthology.orchestra.run.smithi037.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.36820.log.gz 2024-09-17T04:35:16.617 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37178.log 2024-09-17T04:35:16.617 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45985.log.gz 2024-09-17T04:35:16.617 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46137.log 2024-09-17T04:35:16.617 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36822.log: /var/log/ceph/ceph-client.admin.37178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50175.log 2024-09-17T04:35:16.617 INFO:teuthology.orchestra.run.smithi037.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36822.log.gz 2024-09-17T04:35:16.617 INFO:teuthology.orchestra.run.smithi037.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.37178.log.gz 2024-09-17T04:35:16.617 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47320.log 2024-09-17T04:35:16.618 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46137.log.gz 2024-09-17T04:35:16.618 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46263.log 2024-09-17T04:35:16.618 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50175.log: /var/log/ceph/ceph-client.admin.47320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50683.log 2024-09-17T04:35:16.618 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47320.log.gz 2024-09-17T04:35:16.618 INFO:teuthology.orchestra.run.smithi037.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.50175.log.gz 2024-09-17T04:35:16.618 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51490.log 2024-09-17T04:35:16.618 INFO:teuthology.orchestra.run.smithi037.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.0.48708.log.gz 2024-09-17T04:35:16.618 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50683.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46263.log.gz 2024-09-17T04:35:16.618 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51658.log 2024-09-17T04:35:16.618 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50683.log.gz 2024-09-17T04:35:16.618 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42906.log 2024-09-17T04:35:16.618 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51490.log.gz 2024-09-17T04:35:16.618 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47524.log 2024-09-17T04:35:16.619 INFO:teuthology.orchestra.run.smithi037.stderr: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.51658.log.gz 2024-09-17T04:35:16.619 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48000.log 2024-09-17T04:35:16.619 INFO:teuthology.orchestra.run.smithi037.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.42906.log.gz 2024-09-17T04:35:16.619 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43619.log 2024-09-17T04:35:16.619 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47524.log.gz 2024-09-17T04:35:16.619 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48000.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-17T04:35:16.619 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48000.log.gz 2024-09-17T04:35:16.619 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50337.log 2024-09-17T04:35:16.619 INFO:teuthology.orchestra.run.smithi037.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.43619.log.gz 2024-09-17T04:35:16.619 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42678.log 2024-09-17T04:35:16.619 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47651.log 2024-09-17T04:35:16.619 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50337.log.gz 2024-09-17T04:35:16.619 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43244.log 2024-09-17T04:35:16.619 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42678.log.gz 2024-09-17T04:35:16.620 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52437.log 2024-09-17T04:35:16.620 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47651.log: /var/log/ceph/ceph-client.admin.43244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46889.log 2024-09-17T04:35:16.620 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47651.log.gz 2024-09-17T04:35:16.620 INFO:teuthology.orchestra.run.smithi037.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.43244.log.gz 2024-09-17T04:35:16.620 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50217.log 2024-09-17T04:35:16.620 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52437.log.gz 2024-09-17T04:35:16.620 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47141.log 2024-09-17T04:35:16.620 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46889.log.gz 2024-09-17T04:35:16.620 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51448.log 2024-09-17T04:35:16.620 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50217.log: /var/log/ceph/ceph-client.admin.47141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44672.log 2024-09-17T04:35:16.620 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47141.log.gz 2024-09-17T04:35:16.620 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46500.log 2024-09-17T04:35:16.620 INFO:teuthology.orchestra.run.smithi037.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50217.log.gz 2024-09-17T04:35:16.620 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48340.log 2024-09-17T04:35:16.621 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51448.log.gz 2024-09-17T04:35:16.621 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44672.log: /var/log/ceph/ceph-client.admin.46500.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44672.log.gz 2024-09-17T04:35:16.621 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43972.log 2024-09-17T04:35:16.621 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46500.log.gz 2024-09-17T04:35:16.621 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46770.log 2024-09-17T04:35:16.621 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51364.log 2024-09-17T04:35:16.621 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48340.log.gz 2024-09-17T04:35:16.621 INFO:teuthology.orchestra.run.smithi037.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.43972.log.gz 2024-09-17T04:35:16.621 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45821.log 2024-09-17T04:35:16.621 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46770.log.gz 2024-09-17T04:35:16.621 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51699.log 2024-09-17T04:35:16.621 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51364.log.gz 2024-09-17T04:35:16.621 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36279.log 2024-09-17T04:35:16.622 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.2.45821.log: /var/log/ceph/ceph-client.admin.51699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52773.log 2024-09-17T04:35:16.622 INFO:teuthology.orchestra.run.smithi037.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51699.log.gz 2024-09-17T04:35:16.622 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.2.45821.log.gz 2024-09-17T04:35:16.622 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47566.log 2024-09-17T04:35:16.622 INFO:teuthology.orchestra.run.smithi037.stderr: 75.3% -- replaced with /var/log/ceph/ceph-client.admin.36279.log.gz 2024-09-17T04:35:16.622 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46712.log 2024-09-17T04:35:16.622 INFO:teuthology.orchestra.run.smithi037.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52773.log.gz 2024-09-17T04:35:16.622 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51909.log 2024-09-17T04:35:16.622 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47566.log.gz 2024-09-17T04:35:16.622 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43968.log 2024-09-17T04:35:16.622 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46712.log.gz 2024-09-17T04:35:16.622 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50007.log 2024-09-17T04:35:16.623 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51909.log: /var/log/ceph/ceph-client.admin.43968.log: gzip 87.3% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.51909.log.gz -- 2024-09-17T04:35:16.623 INFO:teuthology.orchestra.run.smithi037.stderr: /var/log/ceph/ceph-client.admin.36364.log 2024-09-17T04:35:16.623 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43968.log.gz 2024-09-17T04:35:16.623 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49690.log 2024-09-17T04:35:16.623 INFO:teuthology.orchestra.run.smithi037.stderr:gzip 90.8% -- replaced with /var/log/ceph/ceph-client.admin.50007.log.gz -5 2024-09-17T04:35:16.623 INFO:teuthology.orchestra.run.smithi037.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42862.log 2024-09-17T04:35:16.623 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36364.log: 77.3%/var/log/ceph/ceph-client.admin.49690.log: -- replaced with /var/log/ceph/ceph-client.admin.36364.log.gz 2024-09-17T04:35:16.623 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43970.log 2024-09-17T04:35:16.623 INFO:teuthology.orchestra.run.smithi037.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49690.log.gz 2024-09-17T04:35:16.623 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49101.log 2024-09-17T04:35:16.623 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42862.log.gz 2024-09-17T04:35:16.623 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48088.log 2024-09-17T04:35:16.623 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43970.log: /var/log/ceph/ceph-client.admin.49101.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43970.log.gz 2024-09-17T04:35:16.624 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42820.log 2024-09-17T04:35:16.624 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49101.log.gz 2024-09-17T04:35:16.624 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46931.log 2024-09-17T04:35:16.624 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48088.log.gz 2024-09-17T04:35:16.624 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53275.log 2024-09-17T04:35:16.624 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42820.log: /var/log/ceph/ceph-client.admin.46931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36722.log 2024-09-17T04:35:16.624 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42820.log.gz 2024-09-17T04:35:16.624 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46931.log.gz 2024-09-17T04:35:16.624 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36825.log 2024-09-17T04:35:16.624 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53275.log.gz 2024-09-17T04:35:16.624 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48172.log 2024-09-17T04:35:16.624 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36722.log: /var/log/ceph/ceph-client.admin.36825.log: 85.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.36722.log.gz -5 2024-09-17T04:35:16.624 INFO:teuthology.orchestra.run.smithi037.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43634.log 2024-09-17T04:35:16.625 INFO:teuthology.orchestra.run.smithi037.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36825.log.gz 2024-09-17T04:35:16.625 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43202.log 2024-09-17T04:35:16.625 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36307.log 2024-09-17T04:35:16.625 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48172.log.gz 2024-09-17T04:35:16.625 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43634.log: /var/log/ceph/ceph-client.admin.43202.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.43634.log.gz 2024-09-17T04:35:16.625 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47958.log 2024-09-17T04:35:16.625 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43202.log.gz 2024-09-17T04:35:16.625 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46095.log 2024-09-17T04:35:16.625 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36307.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36307.log.gz 2024-09-17T04:35:16.625 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47609.log 2024-09-17T04:35:16.625 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47958.log.gz 2024-09-17T04:35:16.625 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36826.log 2024-09-17T04:35:16.625 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46095.log.gz 2024-09-17T04:35:16.625 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48130.log 2024-09-17T04:35:16.625 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47609.log: /var/log/ceph/ceph-client.admin.36826.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T04:35:16.626 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47609.log.gz 2024-09-17T04:35:16.626 INFO:teuthology.orchestra.run.smithi037.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.36826.log.gz 2024-09-17T04:35:16.626 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44306.log 2024-09-17T04:35:16.626 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48130.log.gz 2024-09-17T04:35:16.626 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44307.log 2024-09-17T04:35:16.626 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42501.log 2024-09-17T04:35:16.626 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44306.log: /var/log/ceph/ceph-client.admin.44307.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44306.log.gz 2024-09-17T04:35:16.626 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52096.log 2024-09-17T04:35:16.626 INFO:teuthology.orchestra.run.smithi037.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44307.log.gz 2024-09-17T04:35:16.626 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0%/var/log/ceph/ceph-client.admin.42501.log: -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T04:35:16.626 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47015.log 2024-09-17T04:35:16.626 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42501.log.gz 2024-09-17T04:35:16.626 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50575.log 2024-09-17T04:35:16.626 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52521.log 2024-09-17T04:35:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47015.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52096.log.gz 2024-09-17T04:35:16.627 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47015.log.gz 2024-09-17T04:35:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49648.log 2024-09-17T04:35:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50641.log 2024-09-17T04:35:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52521.log: /var/log/ceph/ceph-client.admin.49648.log: gzip -5 --verbose 87.2% -- /var/log/ceph/ceph-client.admin.50533.log -- replaced with /var/log/ceph/ceph-client.admin.52521.log.gz 2024-09-17T04:35:16.627 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-09-17T04:35:16.627 INFO:teuthology.orchestra.run.smithi037.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.50575.log.gz 2024-09-17T04:35:16.627 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49648.log.gz 2024-09-17T04:35:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49522.log 2024-09-17T04:35:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44168.log 2024-09-17T04:35:16.627 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50641.log.gz 2024-09-17T04:35:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51322.log 2024-09-17T04:35:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49522.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50533.log.gz 2024-09-17T04:35:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44168.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49522.log.gz 2024-09-17T04:35:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51280.log 2024-09-17T04:35:16.628 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44168.log.gz 2024-09-17T04:35:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36681.log 2024-09-17T04:35:16.628 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51322.log.gz 2024-09-17T04:35:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49438.log 2024-09-17T04:35:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51280.log: /var/log/ceph/ceph-client.admin.36681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53193.log 2024-09-17T04:35:16.628 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51280.log.gz 2024-09-17T04:35:16.628 INFO:teuthology.orchestra.run.smithi037.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.36681.log.gz 2024-09-17T04:35:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36791.log 2024-09-17T04:35:16.628 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49438.log.gz 2024-09-17T04:35:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36778.log 2024-09-17T04:35:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53193.log: /var/log/ceph/ceph-client.admin.36791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49816.log 2024-09-17T04:35:16.628 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53193.log.gz 2024-09-17T04:35:16.629 INFO:teuthology.orchestra.run.smithi037.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.36791.log.gz 2024-09-17T04:35:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49278.log 2024-09-17T04:35:16.629 INFO:teuthology.orchestra.run.smithi037.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.36778.log.gz 2024-09-17T04:35:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46596.log 2024-09-17T04:35:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49816.log: /var/log/ceph/ceph-client.admin.49278.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49816.log.gz 2024-09-17T04:35:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49163.log 2024-09-17T04:35:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48448.log 2024-09-17T04:35:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46596.log: /var/log/ceph/ceph-client.admin.49163.log: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.49278.log.gz 2024-09-17T04:35:16.629 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46596.log.gz 2024-09-17T04:35:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46347.log 2024-09-17T04:35:16.629 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49163.log.gz 2024-09-17T04:35:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50943.log 2024-09-17T04:35:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48448.log: /var/log/ceph/ceph-client.admin.46347.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48448.log.gz 2024-09-17T04:35:16.630 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43034.log 2024-09-17T04:35:16.630 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46347.log.gz 2024-09-17T04:35:16.630 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.50943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36528.log 2024-09-17T04:35:16.630 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51825.log 2024-09-17T04:35:16.630 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43034.log: /var/log/ceph/ceph-client.admin.36528.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43034.log.gz -5 2024-09-17T04:35:16.630 INFO:teuthology.orchestra.run.smithi037.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50049.log 2024-09-17T04:35:16.630 INFO:teuthology.orchestra.run.smithi037.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.36528.log.gz 2024-09-17T04:35:16.630 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42950.log 2024-09-17T04:35:16.630 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37475.log 2024-09-17T04:35:16.630 INFO:teuthology.orchestra.run.smithi037.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51825.log.gz 2024-09-17T04:35:16.630 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50049.log: /var/log/ceph/ceph-client.admin.42950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52815.log 2024-09-17T04:35:16.630 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42950.log.gz 2024-09-17T04:35:16.631 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44546.log 2024-09-17T04:35:16.631 INFO:teuthology.orchestra.run.smithi037.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.0.50943.log.gz 2024-09-17T04:35:16.631 INFO:teuthology.orchestra.run.smithi037.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.37475.log.gz 2024-09-17T04:35:16.631 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45943.log 2024-09-17T04:35:16.631 INFO:teuthology.orchestra.run.smithi037.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.50049.log.gz 2024-09-17T04:35:16.631 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44546.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52815.log.gzgzip 2024-09-17T04:35:16.631 INFO:teuthology.orchestra.run.smithi037.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47907.log 2024-09-17T04:35:16.631 INFO:teuthology.orchestra.run.smithi037.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.44546.log.gz 2024-09-17T04:35:16.631 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46221.log 2024-09-17T04:35:16.631 INFO:teuthology.orchestra.run.smithi037.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45943.log.gz 2024-09-17T04:35:16.631 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52899.log 2024-09-17T04:35:16.631 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47907.log.gz 2024-09-17T04:35:16.631 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43412.log 2024-09-17T04:35:16.632 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46221.log: /var/log/ceph/ceph-client.admin.52899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36252.log 2024-09-17T04:35:16.632 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46221.log.gz 2024-09-17T04:35:16.632 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52899.log.gz 2024-09-17T04:35:16.632 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47183.log 2024-09-17T04:35:16.632 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43412.log.gz 2024-09-17T04:35:16.632 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45322.log 2024-09-17T04:35:16.632 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36252.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36252.log.gz 2024-09-17T04:35:16.632 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52054.log 2024-09-17T04:35:16.632 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.45322.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47183.log.gz 2024-09-17T04:35:16.632 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48298.log 2024-09-17T04:35:16.632 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52353.log 2024-09-17T04:35:16.632 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52054.log: 89.3% -- replaced with /var/log/ceph/ceph-client.0.45322.log.gz 2024-09-17T04:35:16.633 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51951.log 2024-09-17T04:35:16.633 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52054.log.gz 2024-09-17T04:35:16.633 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48298.log.gz 2024-09-17T04:35:16.633 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46973.log 2024-09-17T04:35:16.633 INFO:teuthology.orchestra.run.smithi037.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52353.log.gz 2024-09-17T04:35:16.633 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-17T04:35:16.633 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51951.log: /var/log/ceph/ceph-client.admin.46973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36643.log 2024-09-17T04:35:16.633 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51951.log.gz 2024-09-17T04:35:16.633 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46973.log.gz 2024-09-17T04:35:16.633 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43969.log 2024-09-17T04:35:16.633 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49858.log 2024-09-17T04:35:16.633 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36643.log: /var/log/ceph/ceph-client.admin.43969.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36643.log.gz 2024-09-17T04:35:16.633 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44756.log 2024-09-17T04:35:16.633 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43969.log.gz 2024-09-17T04:35:16.634 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42582.log 2024-09-17T04:35:16.634 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49858.log.gz 2024-09-17T04:35:16.634 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44798.log 2024-09-17T04:35:16.634 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44756.log: /var/log/ceph/ceph-client.admin.42582.log: gzip -5 --verbose -- 87.1% /var/log/ceph/ceph-client.admin.52605.log 2024-09-17T04:35:16.634 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44756.log.gz 2024-09-17T04:35:16.634 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42582.log.gz 2024-09-17T04:35:16.634 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43118.log 2024-09-17T04:35:16.634 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44300.log 2024-09-17T04:35:16.634 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52605.log.gz 2024-09-17T04:35:16.634 INFO:teuthology.orchestra.run.smithi037.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.44798.log.gz 2024-09-17T04:35:16.634 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45445.log 2024-09-17T04:35:16.634 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43118.log.gz 2024-09-17T04:35:16.634 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52138.log 2024-09-17T04:35:16.635 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44300.log.gz 2024-09-17T04:35:16.635 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47863.log 2024-09-17T04:35:16.635 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45445.log.gz 2024-09-17T04:35:16.635 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43971.log 2024-09-17T04:35:16.635 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52138.log.gz 2024-09-17T04:35:16.635 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44305.log 2024-09-17T04:35:16.635 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47863.log.gz 2024-09-17T04:35:16.635 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47819.log 2024-09-17T04:35:16.635 INFO:teuthology.orchestra.run.smithi037.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.43971.log.gz 2024-09-17T04:35:16.635 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47777.log 2024-09-17T04:35:16.635 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44305.log.gz 2024-09-17T04:35:16.635 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47099.log 2024-09-17T04:35:16.635 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47819.log.gz 2024-09-17T04:35:16.636 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49774.log 2024-09-17T04:35:16.636 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47777.log.gz 2024-09-17T04:35:16.636 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43631.log 2024-09-17T04:35:16.636 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49774.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47099.log.gz 2024-09-17T04:35:16.636 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36195.log 2024-09-17T04:35:16.636 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49774.log.gz 2024-09-17T04:35:16.636 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43631.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-17T04:35:16.636 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43631.log.gz 2024-09-17T04:35:16.636 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37138.log 2024-09-17T04:35:16.636 INFO:teuthology.orchestra.run.smithi037.stderr: 76.5% -- replaced with /var/log/ceph/ceph-client.admin.36195.log.gz 2024-09-17T04:35:16.636 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46442.log 2024-09-17T04:35:16.636 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50284.log 2024-09-17T04:35:16.636 INFO:teuthology.orchestra.run.smithi037.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.37138.log.gz 2024-09-17T04:35:16.636 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47482.log 2024-09-17T04:35:16.637 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46442.log.gz 2024-09-17T04:35:16.637 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51574.log 2024-09-17T04:35:16.637 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50284.log.gz 2024-09-17T04:35:16.637 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44900.log 2024-09-17T04:35:16.637 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47482.log.gz 2024-09-17T04:35:16.637 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51574.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-17T04:35:16.637 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51574.log.gz 2024-09-17T04:35:16.637 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46305.log 2024-09-17T04:35:16.637 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44900.log.gz 2024-09-17T04:35:16.637 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44504.log 2024-09-17T04:35:16.637 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51783.log 2024-09-17T04:35:16.637 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46305.log.gz 2024-09-17T04:35:16.637 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44504.log: gzip -5 --verbose 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44504.log.gz 2024-09-17T04:35:16.638 INFO:teuthology.orchestra.run.smithi037.stderr: -- /var/log/ceph/ceph-client.admin.36582.log 2024-09-17T04:35:16.638 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51783.log: gzip -5 --verbose -- 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51783.log.gz 2024-09-17T04:35:16.638 INFO:teuthology.orchestra.run.smithi037.stderr: /var/log/ceph/ceph-client.admin.44714.log 2024-09-17T04:35:16.638 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36582.log: gzip -5 --verbose 85.6% -- replaced with /var/log/ceph/ceph-client.admin.36582.log.gz 2024-09-17T04:35:16.638 INFO:teuthology.orchestra.run.smithi037.stderr: -- /var/log/ceph/ceph-client.admin.48382.log 2024-09-17T04:35:16.638 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44714.log: gzip -5 --verbose -- 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44714.log.gz 2024-09-17T04:35:16.638 INFO:teuthology.orchestra.run.smithi037.stderr: /var/log/ceph/ceph-client.admin.52395.log 2024-09-17T04:35:16.638 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52012.log 2024-09-17T04:35:16.638 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47267.log 2024-09-17T04:35:16.638 INFO:teuthology.orchestra.run.smithi037.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.48382.log.gz 2024-09-17T04:35:16.638 INFO:teuthology.orchestra.run.smithi037.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.52395.log.gz 2024-09-17T04:35:16.638 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48948.log 2024-09-17T04:35:16.638 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52012.log: /var/log/ceph/ceph-client.admin.47267.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52012.log.gz 2024-09-17T04:35:16.638 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47267.log.gz 2024-09-17T04:35:16.639 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53067.log 2024-09-17T04:35:16.639 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53151.log 2024-09-17T04:35:16.639 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48948.log.gz/var/log/ceph/ceph-client.admin.53067.log: 2024-09-17T04:35:16.639 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53067.log.gz 2024-09-17T04:35:16.639 INFO:teuthology.orchestra.run.smithi037.stderr: -- /var/log/ceph/ceph-client.admin.46847.log 2024-09-17T04:35:16.639 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53151.log: gzip -5 --verbose -- 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53151.log.gz 2024-09-17T04:35:16.639 INFO:teuthology.orchestra.run.smithi037.stderr: /var/log/ceph/ceph-client.admin.53233.log 2024-09-17T04:35:16.639 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46847.log: gzip -5 --verbose -- 87.7% /var/log/ceph/ceph-client.admin.43160.log 2024-09-17T04:35:16.639 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46847.log.gz 2024-09-17T04:35:16.639 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43633.log 2024-09-17T04:35:16.639 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53233.log.gz 2024-09-17T04:35:16.639 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-17T04:35:16.639 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43160.log: /var/log/ceph/ceph-client.admin.43633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49392.log 2024-09-17T04:35:16.640 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-mgr.y.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43160.log.gz 2024-09-17T04:35:16.640 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43633.log.gz 2024-09-17T04:35:16.640 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51741.log 2024-09-17T04:35:16.640 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52983.log 2024-09-17T04:35:16.640 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49392.log: /var/log/ceph/ceph-client.admin.51741.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49392.log.gz 2024-09-17T04:35:16.640 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51741.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36336.log 2024-09-17T04:35:16.640 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-09-17T04:35:16.640 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43959.log 2024-09-17T04:35:16.640 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52983.log: /var/log/ceph/ceph-client.admin.36336.log: 78.0% -- replaced with /var/log/ceph/ceph-client.admin.36336.log.gz 2024-09-17T04:35:16.640 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52983.log.gz 2024-09-17T04:35:16.640 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T04:35:16.640 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44308.log 2024-09-17T04:35:16.640 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43959.log.gz 2024-09-17T04:35:16.640 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- 92.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-17T04:35:16.640 INFO:teuthology.orchestra.run.smithi037.stderr: /var/log/ceph/ceph-client.admin.51406.log 2024-09-17T04:35:16.641 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50133.log 2024-09-17T04:35:16.641 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44308.log.gz 2024-09-17T04:35:16.641 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51406.log: 85.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T04:35:16.641 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36823.log 2024-09-17T04:35:16.641 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51406.log.gz 2024-09-17T04:35:16.641 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43625.log 2024-09-17T04:35:16.641 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50133.log: /var/log/ceph/ceph-client.admin.36823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52941.log 2024-09-17T04:35:16.641 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43625.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36823.log.gz 2024-09-17T04:35:16.641 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43625.log.gz 2024-09-17T04:35:16.641 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42778.log 2024-09-17T04:35:16.641 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49006.log 2024-09-17T04:35:16.641 INFO:teuthology.orchestra.run.smithi037.stderr: 92.4%/var/log/ceph/ceph-client.admin.42778.log: -- replaced with /var/log/ceph/ceph-client.admin.50133.log.gz 2024-09-17T04:35:16.641 INFO:teuthology.orchestra.run.smithi037.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52941.log.gz 2024-09-17T04:35:16.641 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42778.log.gz 2024-09-17T04:35:16.641 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43286.log 2024-09-17T04:35:16.642 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53109.log 2024-09-17T04:35:16.642 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49006.log: gzip/var/log/ceph/ceph-client.admin.43286.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48256.log 2024-09-17T04:35:16.642 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1%/var/log/ceph/ceph-client.admin.53109.log: -- replaced with /var/log/ceph/ceph-client.admin.43286.log.gz 2024-09-17T04:35:16.642 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49006.log.gz 2024-09-17T04:35:16.642 INFO:teuthology.orchestra.run.smithi037.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53109.log.gz 2024-09-17T04:35:16.642 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36489.log 2024-09-17T04:35:16.642 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43965.log 87.0% -- replaced with /var/log/ceph/ceph-client.admin.48256.log.gz 2024-09-17T04:35:16.642 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-09-17T04:35:16.642 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36489.log: gzip -5 --verbose 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36489.log.gz 2024-09-17T04:35:16.642 INFO:teuthology.orchestra.run.smithi037.stderr: -- /var/log/ceph/ceph-client.admin.52311.log 2024-09-17T04:35:16.643 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49881.log 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43965.log.gz 2024-09-17T04:35:16.643 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-09-17T04:35:16.643 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52311.log: gzip -5 --verbose 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52311.log.gz 2024-09-17T04:35:16.643 INFO:teuthology.orchestra.run.smithi037.stderr: -- /var/log/ceph/ceph-client.admin.47401.log 2024-09-17T04:35:16.643 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53025.log 2024-09-17T04:35:16.643 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47401.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.49881.log.gz 2024-09-17T04:35:16.643 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49564.log 2024-09-17T04:35:16.643 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47401.log.gz 2024-09-17T04:35:16.643 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52647.log 2024-09-17T04:35:16.643 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49480.log 2024-09-17T04:35:16.643 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49564.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53025.log.gz 2024-09-17T04:35:16.643 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4%/var/log/ceph/ceph-client.admin.52647.log: -- replaced with /var/log/ceph/ceph-client.admin.49564.log.gz 2024-09-17T04:35:16.643 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43624.log 2024-09-17T04:35:16.643 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52689.log 2024-09-17T04:35:16.644 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52647.log.gz 2024-09-17T04:35:16.644 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43624.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49480.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43454.log 2024-09-17T04:35:16.644 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-09-17T04:35:16.644 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43624.log.gz 2024-09-17T04:35:16.644 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49606.log 2024-09-17T04:35:16.644 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52689.log: /var/log/ceph/ceph-client.admin.43454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50379.log 2024-09-17T04:35:16.644 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52689.log.gz 2024-09-17T04:35:16.644 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43454.log.gz 2024-09-17T04:35:16.644 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49606.log: gzip -5 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49606.log.gz 2024-09-17T04:35:16.644 INFO:teuthology.orchestra.run.smithi037.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44588.log 2024-09-17T04:35:16.644 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37380.log 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50379.log.gz 2024-09-17T04:35:16.644 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-09-17T04:35:16.644 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44588.log: gzip -5 --verbose -- 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44588.log.gz 2024-09-17T04:35:16.644 INFO:teuthology.orchestra.run.smithi037.stderr: /var/log/ceph/ceph-client.admin.42992.log 2024-09-17T04:35:16.645 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37380.log: gzip -5 86.8% -- replaced with /var/log/ceph/ceph-client.admin.37380.log.gz 2024-09-17T04:35:16.645 INFO:teuthology.orchestra.run.smithi037.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51616.log 2024-09-17T04:35:16.645 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43636.log 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42992.log.gz 2024-09-17T04:35:16.645 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-09-17T04:35:16.645 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51616.log: gzip -5 --verbose 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51616.log.gz 2024-09-17T04:35:16.645 INFO:teuthology.orchestra.run.smithi037.stderr: -- /var/log/ceph/ceph-client.admin.44630.log 2024-09-17T04:35:16.645 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36745.log 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43636.log.gz 2024-09-17T04:35:16.645 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-09-17T04:35:16.645 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44630.log: gzip -5 --verbose 87.4% -- /var/log/ceph/ceph-client.admin.50091.log 2024-09-17T04:35:16.645 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44630.log.gz 2024-09-17T04:35:16.645 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36745.log: gzip -5 --verbose 85.4% -- replaced with /var/log/ceph/ceph-client.admin.36745.log.gz 2024-09-17T04:35:16.645 INFO:teuthology.orchestra.run.smithi037.stderr: -- /var/log/ceph/ceph-client.admin.43076.log 2024-09-17T04:35:16.645 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49923.log 2024-09-17T04:35:16.645 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52857.log 2024-09-17T04:35:16.645 INFO:teuthology.orchestra.run.smithi037.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.43076.log.gz 2024-09-17T04:35:16.646 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48214.log 2024-09-17T04:35:16.646 INFO:teuthology.orchestra.run.smithi037.stderr: 92.3%/var/log/ceph/ceph-client.admin.52857.log: -- replaced with /var/log/ceph/ceph-client.admin.50091.log.gz 2024-09-17T04:35:16.646 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% 90.4% -- replaced with /var/log/ceph/ceph-client.admin.49923.log.gz 2024-09-17T04:35:16.646 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52857.log.gz 2024-09-17T04:35:16.646 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36394.log 2024-09-17T04:35:16.646 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47057.log 2024-09-17T04:35:16.646 INFO:teuthology.orchestra.run.smithi037.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.48214.log.gz/var/log/ceph/ceph-client.admin.36394.log: 2024-09-17T04:35:16.646 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43963.log 2024-09-17T04:35:16.646 INFO:teuthology.orchestra.run.smithi037.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.36394.log.gz 2024-09-17T04:35:16.646 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5/var/log/ceph/ceph-client.admin.47057.log: --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-17T04:35:16.646 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43963.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47057.log.gz 2024-09-17T04:35:16.646 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52731.log 2024-09-17T04:35:16.646 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43963.log.gz 2024-09-17T04:35:16.646 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-17T04:35:16.646 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.52731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49732.log 2024-09-17T04:35:16.647 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-mds.c.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52731.log.gz 2024-09-17T04:35:16.647 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46179.log 2024-09-17T04:35:16.647 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49732.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49732.log.gz 2024-09-17T04:35:16.647 INFO:teuthology.orchestra.run.smithi037.stderr:gzip 92.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-17T04:35:16.647 INFO:teuthology.orchestra.run.smithi037.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44299.log 2024-09-17T04:35:16.647 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50467.log 2024-09-17T04:35:16.647 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46179.log.gz 2024-09-17T04:35:16.647 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43370.log 2024-09-17T04:35:16.647 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44299.log.gz 2024-09-17T04:35:16.647 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50467.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.50467.log.gz 2024-09-17T04:35:16.647 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47693.log 2024-09-17T04:35:16.647 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44303.log 2024-09-17T04:35:16.647 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43370.log.gz 2024-09-17T04:35:16.647 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47693.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30151.log 2024-09-17T04:35:16.647 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44303.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47693.log.gz 2024-09-17T04:35:16.648 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44303.log.gz 2024-09-17T04:35:16.648 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47225.log 2024-09-17T04:35:16.648 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph.tmp-client.admin.30151.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30151.log.gz 2024-09-17T04:35:16.648 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48042.log 2024-09-17T04:35:16.648 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51867.log 2024-09-17T04:35:16.648 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48042.log: /var/log/ceph/ceph-client.admin.47225.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48042.log.gz 2024-09-17T04:35:16.648 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47225.log.gz 2024-09-17T04:35:16.648 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52479.log 2024-09-17T04:35:16.648 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52563.log 2024-09-17T04:35:16.648 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51867.log.gz 2024-09-17T04:35:16.648 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52479.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52479.log.gz/var/log/ceph/ceph-client.admin.52563.log: 2024-09-17T04:35:16.648 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52563.log.gz 2024-09-17T04:35:16.649 INFO:teuthology.orchestra.run.smithi037.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-17T04:35:16.649 INFO:teuthology.orchestra.run.smithi092.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-17T04:35:16.653 INFO:teuthology.orchestra.run.smithi037.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-17T04:35:16.672 INFO:teuthology.orchestra.run.smithi092.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-17T04:35:16.674 INFO:teuthology.orchestra.run.smithi092.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-17T04:35:16.675 INFO:teuthology.orchestra.run.smithi037.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-17T04:35:16.680 INFO:teuthology.orchestra.run.smithi037.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-17T04:35:16.692 INFO:teuthology.orchestra.run.smithi092.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-17T04:35:16.707 INFO:teuthology.orchestra.run.smithi092.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-17T04:35:16.713 INFO:teuthology.orchestra.run.smithi092.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-17T04:35:16.715 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-09-17T04:35:16.715 INFO:teuthology.orchestra.run.smithi092.stderr:real 0m0.192s 2024-09-17T04:35:16.715 INFO:teuthology.orchestra.run.smithi092.stderr:user 0m0.908s 2024-09-17T04:35:16.715 INFO:teuthology.orchestra.run.smithi092.stderr:sys 0m0.059s 2024-09-17T04:35:16.777 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-17T04:35:16.782 INFO:teuthology.orchestra.run.smithi037.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-17T04:35:16.784 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-09-17T04:35:16.784 INFO:teuthology.orchestra.run.smithi037.stderr:real 0m0.283s 2024-09-17T04:35:16.784 INFO:teuthology.orchestra.run.smithi037.stderr:user 0m0.948s 2024-09-17T04:35:16.784 INFO:teuthology.orchestra.run.smithi037.stderr:sys 0m0.212s 2024-09-17T04:35:16.784 INFO:tasks.ceph:Archiving logs... 2024-09-17T04:35:16.784 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908540/remote/smithi037/log 2024-09-17T04:35:16.786 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T04:35:17.144 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908540/remote/smithi092/log 2024-09-17T04:35:17.145 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T04:35:17.300 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-17T04:35:17.310 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-17T04:35:17.310 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T04:35:17.316 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T04:35:17.399 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-17T04:35:17.407 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-17T04:35:17.408 DEBUG:teuthology.orchestra.run.smithi037:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T04:35:17.410 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T04:35:17.424 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-09-17T04:35:17.425 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-09-17T04:35:17.428 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T04:35:17.428 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-09-17T04:35:17.429 INFO:teuthology.orchestra.run.smithi037.stdout:^* hv01.front.sepia.ceph.com 3 6 377 20 +1552us[+1554us] +/- 59ms 2024-09-17T04:35:17.429 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 20 +247us[ +249us] +/- 74ms 2024-09-17T04:35:17.429 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 -2600us[-2598us] +/- 87ms 2024-09-17T04:35:17.429 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T04:35:17.430 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T04:35:17.430 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-09-17T04:35:17.430 INFO:teuthology.orchestra.run.smithi092.stdout:^* hv01.front.sepia.ceph.com 3 6 377 20 +1548us[+1556us] +/- 59ms 2024-09-17T04:35:17.430 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 20 +256us[ +256us] +/- 74ms 2024-09-17T04:35:17.431 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 -2600us[-2592us] +/- 87ms 2024-09-17T04:35:17.431 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T04:35:17.431 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-17T04:35:17.448 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-17T04:35:17.449 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-17T04:35:17.490 DEBUG:teuthology.orchestra.run.smithi037:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T04:35:17.563 DEBUG:teuthology.orchestra.run.smithi092:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T04:35:17.628 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T04:35:17.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T04:35:17.659 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T04:35:17.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T04:35:17.687 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-17T04:35:17.696 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-17T04:35:17.705 INFO:teuthology.task.internal:Duration was 514.591792 seconds 2024-09-17T04:35:17.705 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-17T04:35:17.713 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-17T04:35:17.714 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T04:35:17.716 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T04:35:17.755 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T04:35:17.766 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T04:35:18.107 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-17T04:35:18.107 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi037.front.sepia.ceph.com 2024-09-17T04:35:18.107 DEBUG:teuthology.orchestra.run.smithi037:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T04:35:18.169 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2024-09-17T04:35:18.170 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T04:35:18.193 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-17T04:35:18.193 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T04:35:18.212 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T04:35:18.259 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-17T04:35:18.259 DEBUG:teuthology.orchestra.run.smithi037:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T04:35:18.279 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T04:35:18.401 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-17T04:35:18.411 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-17T04:35:18.411 DEBUG:teuthology.orchestra.run.smithi037:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T04:35:18.437 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T04:35:18.461 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-17T04:35:18.471 DEBUG:teuthology.orchestra.run.smithi037:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T04:35:18.479 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T04:35:18.500 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = core 2024-09-17T04:35:18.524 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2024-09-17T04:35:18.538 DEBUG:teuthology.orchestra.run.smithi037:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T04:35:18.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T04:35:18.570 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T04:35:18.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T04:35:18.592 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-17T04:35:18.602 INFO:teuthology.task.internal:Transferring archived files... 2024-09-17T04:35:18.603 DEBUG:teuthology.misc:Transferring archived files from smithi037:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908540/remote/smithi037 2024-09-17T04:35:18.603 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T04:35:18.645 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908540/remote/smithi092 2024-09-17T04:35:18.645 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T04:35:18.678 INFO:teuthology.task.internal:Removing archive directory... 2024-09-17T04:35:18.679 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T04:35:18.682 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T04:35:18.728 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-17T04:35:18.738 INFO:teuthology.task.internal:Not uploading archives. 2024-09-17T04:35:18.738 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-17T04:35:18.791 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-17T04:35:18.792 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T04:35:18.794 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T04:35:18.807 INFO:teuthology.orchestra.run.smithi037.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 04:35 /home/ubuntu/cephtest 2024-09-17T04:35:18.809 INFO:teuthology.orchestra.run.smithi092.stdout: 265972 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 04:35 /home/ubuntu/cephtest 2024-09-17T04:35:18.810 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-17T04:35:18.833 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-17T04:35:18.904 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/alternate-pool} duration: 514.5917918682098 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-17T04:35:18.904 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T04:35:18.979 INFO:teuthology.run:pass